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Abstract 


This  thesis  explores  the  effectiveness  of  the  Air  Force  Academy’s  Department  of 
English  local  area  network  peer  feedback  system.  Four  main  questions  were  examined. 
These  questions  explored  a)  whether  students  reported  improvement  in  their  writing,  b) 
whether  students  reported  that  feedback  is  worthwhile,  c)  whether  students  reported  that 
LAN  sessions  increase  enjoyment  of  writing,  and  d)  what  types  of  feedback  the  students 
1)  received  and  2)  found  most  helpful. 

Three  groups  were  studied.  Forty-six  students  received  feedback  online,  fifty 
students  received  feedback  face-to-face,  and  thirty-three  students  did  not  receive 
feedback.  Comparisons  were  made  for  the  questions  mentioned  above  to  see  if  there  was 
any  difference  in  the  responses  based  on  the  method  of  feedback  the  students  received. 

The  results  of  the  study  suggest  that  students’  perceived  writing  improvement  is 
not  affected  by  the  peer  feedback  program.  Peer  feedback  is  successful  in  that  students 
who  participate  find  it  to  be  worthwhile  and  at  least  one  group  believes  that  it  makes 
writing  more  enjoyable.  Finally,  the  two  feedback  groups  reported  receiving  and  wanting 
different  types  of  feedback.  The  online  group  receives  and  prefers  feedback  on  content, 
and  the  face-to-face  group  receives  and  prefers  feedback  on  flow. 


AN  ANALYSIS  OF  THE  EFFECTIVENESS  OF  ONLINE  PEER  FEEDBACK  AT 


THE  UNITED  STATES  AIR  FORCE  ACADEMY 


I.  Introduction 


Chapter  Overview 

There  are  many  different  methods  for  teaching  students  to  write  well.  One  such 
method  is  currently  in  use  at  the  United  States  Air  Force  Academy.  The  Academy’s 
English  Department  has  combined  the  use  of  available  computer  technology  and  the  use 
of  peer  feedback  to  create  a  program  using  the  Academy’s  Local  Area  Network  to 
accomplish  Online  Peer  Feedback.  This  thesis  evaluates  the  effectiveness  of  the  Local 
Area  Network  Online  Peer  Feedback  program.  The  general  research  question  is  whether 
or  not  the  online  peer  feedback  sessions  help  the  students  become  better  writers. 

Background 

The  United  States  Air  Force  Academy  has,  since  its  inception,  required  all 
freshmen  students  to  enroll  in  English  111  (Freshman  English).  The  purpose  of  this  class 
is  to  acquaint  the  students  with  great  literature,  strengthen  their  writing  skills,  and 
enhance  their  analytical  skills.  As  future  leaders  in  the  United  States  Air  Force,  these 
students  must  acquire  the  skill  necessary  in  the  writing-intensive  environment  of  today’s 
Air  Force. 

Until  the  1995-1996  school  year,  a  student’s  final  grade  in  English  111  was  based 
on  three  written  papers  of  8-  to  10-pages  in  length.  These  papers  were  responses  by  the 
students  to  literature  read  as  homework  and  discussed  in  class.  In  1995,  the  Academy 
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English  Department  decided  to  take  advantage  of  the  computer  technology  that  was 
available  to  the  students  and  staff.  This  technology  includes  a  local  area  network  (LAN) 
with  a  computer  in  each  student’s  room  and  on  each  instructor’s  desk.  An  experimental 
curriculvim  incorporating  online  peer  feedback  sessions  using  the  LAN  was  designed  and 
implemented  during  the  1995-1996  academic  year.  Because  faculty  and  students  use  the 
Local  Area  Network,  these  peer  feedback  sessions  are  referred  to  by  the  faculty  as  “LAN 
sessions.”  LAN  sessions  allow  the  students  not  only  to  write  and  edit  using  modem  word 
processing  technology,  but  also  to  critique  each  other’s  work  and  offer  suggestions  for 
improvement. 

Research  Questions 

To  evaluate  the  effectiveness  of  the  online  peer  feedback  curriculum,  members  of 
the  United  States  Air  Force  Academy  English  Department  asked  for  an  answer  to  the 
principal  question,  “do  the  online  peer  feedback  sessions  improve  the  students’  writing?” 
This  thesis  was  designed  around  the  follovting  constraints:  a)  LAN  feedback  sessions 
require  students  to  write  considerably  more  than  was  written  in  the  class  in  previous 
years,  and  b)  no  two  instructors  run  the  LAN  sessions  using  the  same  procedures.  Thus, 
this  study  was  forced  to  rely  on  perceptual  data  about  feedback  effectiveness  gathered 
from  the  students. 

One  of  the  main  differences  in  how  instmctors  implemented  feedback  in  their 
classes  was  the  type  of  feedback  received.  Each  student  fell  into  one  of  three  groups. 

The  first  group  received  peer  feedback  online;  the  second  group  received  peer  feedback 
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face-to-face;  and  the  third  group  did  not  receive  peer  feedback  at  all.  The  research 
questions  to  be  answered  are: 

1 .  Will  students  who  received  peer  feedback,  either  online  or  face-to-face,  report 
more  improvement  in  writing  skills  than  students  who  did  not  participate  in 
feedback? 

2.  Will  students  who  participate  in  online  peer  feedback  exhibit  a  stronger  belief  that 
LAN  sessions  make  writing  more  enjoyable  than  students  who  participate  in  face- 
to-face  feedback? 

3.  Will  students  who  participate  in  peer  feedback  report  that  feedback  is  worthwhile? 

Importance  of  this  Research 

This  information  will  be  useful  to  the  United  States  Air  Force  Academy  English 
Department,  as  it  will  help  in  decision  making  about  the  future  use  of  the  Online  Peer 
Feedback  LAN  sessions.  If  this  research  shows  that  the  students  believe  peer  feedback  is 
helping  them  become  better  writers  and  that  they  enjoy  writing  in  the  local  area  network 
environment,  a  basis  of  support  for  the  spread  of  this  educational  and  motivational  tool 
will  be  established.  This  research  may  be  used  as  guidance  for  making  decisions  about 
requirements  for  classes  other  than  basic  English  to  increase  the  required  writing  as  part 
of  their  course  work  (also  known  as  writing  across  the  curriculum).  This  is  especially 
important  when  one  takes  into  consideration  that  teachers  of  other  subjects  would  have  to 
assign  essays  in  addition  to  teaching  their  regular  coursework.  Peer  Feedback  may  prove 
to  be  a  useful  tool  which  enables  instructors  to  accomplish  everything  they  need  to  do 
without  adding  significant  additional  work  to  their  already  heavy  load. 
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Limitations 


Only  first  year  students  at  the  Air  Force  Academy  were  studied.  Results  from  this 
study  should  be  generalized  across  other  populations  with  extreme  care.  Students  at  the 
academy  exist  in  an  environment  significantly  different  from  the  traditional  college 
environment.  These  students  are  expected  to  maintain  academic,  dress,  behavior,  athletic, 
and  leadership  standards  not  foimd  in  the  typical  college  environment.  However,  this 
research  should  provide  a  starting  point  for  other  researchers  interested  in  online  peer 
feedback  and  its  perceived  worth  to  students. 

Thesis  Overview 

Chapter  I  has  outlined  the  background  for  this  thesis,  summarized  what  research 
questions  are  to  be  answered,  and  discussed  why  this  research  is  important.  Chapter  II 
explores  the  literature  addressing  both  the  use  of  peer  editing  as  a  tool  for  improving 
student’s  writing,  and  the  use  of  computers  in  writing  and  peer  editing.  Chapter  III 
discusses  the  methodology  used  in  this  study  in  depth  and  explains  the  methods  used  by 
the  Academy  English  Department  for  its  particular  brand  of  Peer  Feedback.  Chapter  IV 
shows  the  data  and  gives  the  results  of  each  research  question.  Chapter  V  summarises  the 
results  and  provides  suggestions  for  further  research. 
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II.  Literature  Review 


Introduction 

To  better  understand  the  broad  topic  of  peer  feedback,  the  concept  of  writing  as  a 
process  must  be  understood.  Comprehension  centers  on  the  answers  to  six  questions. 
These  are:  What  is  peer  feedback?  What  are  the  positive  arguments  for  using  peer 
feedback?  What  are  the  concerns  and  drawbacks  to  using  peer  feedback?  What  can  be 
done  to  alleviate  these  concerns  and  drawbacks?  How  is  peer  feedback  implemented? 
And,  how  do  computers  fit  into  the  picture?  This  literature  review  examines  writing  as  a 
process,  then  it  addresses  each  of  these  questions. 

Writing  as  a  Process 

There  is  a  consensus  in  the  literature  that  writing  as  an  academic  exercise  is  no 
longer  viewed  as  the  creation  of  a  product;  rather,  the  emphasis  is  now  on  the  process 
followed  to  create  the  written  product.  Nelson  and  Murphy  (1993:135)  state  that  there  is 
a  “shift  in  the  teaching  of  composition  from  an  emphasis  on  product  to  an  emphasis  on 
process.”  Writing  as  a  process  includes  several  steps,  the  fundamental  building  block  of 
which  is  revision.  Each  paper  should  be  revised  many  times,  each  revision  making  the 
ideas  more  clear  and  the  writing  more  concise.  For  example,  Marcoulides  and  Simkin 
(1991:80)  say  writing  should  be  viewed  as  a  process  “that  requires  planning,  execution, 
and  revision.”  Similarly,  Mangelsdorf  (1992:274)  describes  an  approach  “in  which 
writing  is  seen  as  a  process  of  discovering  and  revising  ideas.”  And  finally,  Healy  and 
Jensen  (1996:58)  describe  a  writing  process  where  “most  writers  benefit  from  writing 
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more  than  one  draft,  from  getting  feedback  on  their  writing,  and  from  having  a  chance  to 
discuss  their  writing  with  interested  readers.” 

The  change  from  writing  as  the  creation  of  a  product  to  writing  as  a  process  can 
sometimes  be  difficult  to  achieve,  but  is  worth  it.  Sultan  (1988)  tells  of  frustration  in 
trying  to  change  the  writing  habits  of  a  group  of  students.  They  had  viewed  writing  as 
the  creation  of  a  product  and  were  ignoring  content  and  placing  the  importance  on 
neatness,  perfect  formation  of  letters,  and  perfect  margins.  Sultan  wanted  them  to  shift  to 
a  program  of  process  writing  where  the  emphasis  is  on  content.  The  basic  difficulty  was 
in  making  the  students  understand  that  revision  is  a  process  that  should  be  aimed  at 
improving  their  commimication  of  an  idea  rather  than  the  idea  that  revision  is  a  process  of 
“recopying”  to  improve  neatness.  Hughes  (1991 :42)  encourages  her  students  to  view 
writing  “as  a  succession  of  small  achievable  steps.” 

What  is  Peer  Feedback? 

While  it  is  not  clearly  stated  in  the  literature,  there  seems  to  be  a  basic  assumption 
that  teaching  writing  as  a  process  requires  a  different  set  of  techniques  than  teaching 
writing  with  an  emphasis  on  product.  A  different  technique  for  “process  writing”  makes 
common  sense.  Process  writing  emphasizes  what  the  writer  is  trying  to  say,  or  the 
content  of  the  paper,  rather  than  the  technical  aspects  of  writing  such  as  spelling  and 
punctuation.  Sultan  (1988),  after  successfully  refocusing  her  students  from  a  product 
emphasis  to  a  process  emphasis,  saw  94  per  cent  of  her  students  pass  the  Mississippi 
State  Fimctional  Literacy  Exam,  a  statewide  test  with  a  direct  writing  sample.  Statewide, 
students  like  hers  averaged  only  a  54  percent  pass  rate.  She  claims  that  grammar  drill. 


6 


emphasis  on  basic  skills  using  workbooks,  and  revision  based  on  teacher  evaluation  of 
drafts  are  not  responsible,  as  these  techniques  were  not  used  in  her  classroom.  Rather, 
her  emphasis  was  on  writing  for  content  and  participation  “in  peer  response  groups  for 
content  feedback”  (Sultan,  1988:66).  MacArthur  (1994:25-26)  says  that  the  success  of 
the  process  approach  may  result  in  part  because  the  teacher  “responds  to  what  the  student 
has  to  say  before  working  on  skill  development.”  In  MacArthur’s  classroom,  peer 
evaluators  are  encouraged  to  focus  on  content  and  communication  first,  and  not  spelling 
and  grammar  errors.  Katstra,  Tollefson  and  Gilbert  (1987:168)  claim  that  emphasis  on 
mechanics  and  spelling  coupled  with  the  students’  knowledge  that  what  they  write  will  be 
corrected  in  the  “traditional  red  ink”  can  lead  to  the  replacement  of  enthusiasm  for 
writing  with  worry  over  form. 

One  technique  to  improve  writing  using  the  process  approach  is  to  expose  the 
students’  writing  to  the  criticism  of  their  peers.  This  technique  is  being  heard  about  more 
often,  both  in  the  classroom  and  in  the  literature.  As  early  as  1972,  Keimeth  Bruffee 
published  a  textbook  “that  has  served  as  the  primary  college  textbook  for  teaching  writing 
using  collaborative  learning”  (Holt,  1992:386).  Unfortunately,  his  technique  was  not 
embraced  as  widely  as  it  might  have  been.  It  was  not  until  almost  two  decades  later,  in 
the  late  1980s,  that  peer  feedback  seemed  to  come  into  its  own. 

Lardner  (1989)  describes  the  use  of  peer  response  groups  in  the  process  approach 
to  writing  as  the  most  promising.  Although  the  term  peer  feedback  is  highly  descriptive 
and  is  virtually  self-explanatory,  it  is  known  by  many  different  names  in  the  literature: 
peer  feedback  (Elbow  and  Belanoff,  1995),  peer  response  (Nelson,  1993,  Holt,  1992), 
collaborative  revision  (Sultan,  1988),  peer  review  (Hughes,  1991),  peer  evaluation 
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(Liftig,  1990)  and  peer  critique  (Holt,  1992).  Whatever  the  name,  they  all  agree  on  one 
point;  peer  feedback  is  invaluable  in  helping  students  to  write  better. 

Positive  Arguments  for  Using  Peer  Feedback 

A  main  goal  of  any  English  class  is  to  improve  the  writing  skills  of  the  students  in 
that  class.  Research  indicates  that  peer  feedback  is  an  effective  way  to  achieve  this 
improvement  (Healy  and  Jensen,  1996;  Hughes,  1991;  Elbow  and  Belanoff,  1995; 
Nelson,  1993;  Sultan,  1988Liftig,  1990;  Harp,  1988;  Lardner,  1989;  Davis,  1996; 
Tollefson  and  Gilbert,  1987).  Lardner  (1989)  comments  that  participation  in  peer 
response  results  in  stronger  revisions  than  participation  in  student  self-evaluation.  This 
comment  means  that  students  make  more  changes  and  write  better  papers  based  on  peer 
response.  Research  suggests  that  this  improvement  can  be  accomplished  by  the  act  of 
revising  the  current  paper,  or  by  internalizing  the  feedback  and  showing  improvement  in 
future  writing  (Sultan,  1988).  Harp  (1988)  adds  that  there  is  evidence  that  reading 
comprehension  can  be  improved  because  of  peer  editing. 

Sultan  (1988)  observes  that  her  students  began  discussing  their  papers  with  their 
peer  groups  even  while  they  were  still  in  the  planning  stage.  Because  the  concept  of  peer 
review  was  introduced  and  encouraged,  the  students  were  comfortable  with  the  peer 
feedback  process.  Students  were  then  more  apt  to  ask  for  help  from  their  classmates  on  a 
casual  basis,  thus  giving  more  thought  and  more  time  to  each  essay  even  before  the  first 
word  was  written.  She  also  notes  that  there  was  improvement  in  their  writing  skills  from 
paper  to  paper  in  spite  of  a  reluctance  to  accomplish  numerous  drafts.  This  is  interesting 
to  note  in  light  of  the  fact  that  she  also  states  she  “cannot  support  that  more  and  better 
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revision  of  individual  drafts  occurred”  (Sultan,  1988:66).  Sultan  suggests  that  there  may 
be  improvement  in  writing  skills  just  by  participating  in  the  peer  groups,  regardless  of  the 
effort  put  into  the  actual  writing. 

Hughes  (1991)  observed  improvement  in  her  students’  writing  based  not  on  peer 
feedback,  but  on  their  participation  in  a  program  that  required  them  to  actively  tutor 
students  from  a  lower  grade  level.  This  observation  is  related  to  Schramm  and  Rich’s 
(1990:36)  observation  that  students  learn  more  from  “participating  in  the  learning 
experience  than  by  watching  it  happen.”  Specific  to  peer  review,  Schramm  and  Rich 
(1990:36)  observe  that  evaluating  their  peers  helps  students  “better  understand  what  is 
expected  of  themselves.”  Because  students  are  actively  participating,  they  have  to  pay 
closer  attention  to  the  material  and  thus  are  in  a  better  position  to  imderstand  and 
remember  it. 

Schramm  and  Rich  (1990)  also  observe  that  receiving  feedback  from  their  peers 
gives  the  student  information  from  several  different  points  of  view,  rather  than  only  one 
point  of  view,  that  of  the  instructor.  The  understanding  of  different  points  of  view  is 
important  as  it  lets  student  authors  recognize  and  write  for  different  audiences 
(Mangelsdorf,  1992,  Healy  and  Jensen,  1996,  Nelson,  1993,  Holt,  1992).  Holt 
(1992:390)  also  suggests  that  feedback  from  different  points  of  view  may  benefit  the 
student  later  in  life,  as  the  review  of  written  work  by  several  different  people  from 
different  backgrounds  is  “similar  to  the  way  professional  writing  may  be  scrutinized.” 
Mangelsdorf  (1992)  comments  that  peer  reviews  make  the  audience  real,  giving  the 
author  the  ability  to  see  how  that  real  audience  will  respond  to  what  was  written. 
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On  the  other  hand,  while  Newkirk  (1984)  agrees  that  peer  feedback  leads  to 
several  different  points  of  view,  he  identifies  this  recognition  of  different  points  of  view 
as  a  problem.  If  the  students  are  ultimately  graded  by  an  instructor,  are  being  told  that 
their  papers  should  be  written  for  a  peer  audience,  and  are  receiving  all  interim  feedback 
from  a  peer  group,  Newkirk  (1984)  questions  whether  or  not  the  evaluation  of  the  peer 
group  will  be  consistent  with  the  goals  and  evaluation  of  the  instructor.  He  finds  that 
students  and  instructors  in  his  study  use  different  criteria  when  evaluating  student  work. 
He  suggests  that  the  common  practice  of  telling  students  to  aim  their  papers  at  a  peer 
audience  should  not  be  continued,  especially  when  the  ultimate  grade  will  not  be  assigned 
by  this  same  peer  group.  While  this  study  does  not  support  one  of  the  popular  reasons  for 
using  peer  review  as  a  classroom  tool,  it  does  agree  with  the  use  of  peer  feedback  and 
gives  a  different  reason: 

Students  need  practice  applying  the  criteria  that  they  are  now  learning. 

But  rather  than  being  viewed  as  the  “natural”  audience  for  fellow-students’ 
writing,  they  might  more  profitably  be  viewed  as  apprentices,  attempting 
to  learn  and  apply  criteria  appropriate  to  an  academic  audience.  (Newkirk, 
1984:310) 

One  of  the  most  practical  reasons  to  use  peer  feedback  is  the  time  that  it 
can  save  an  instructor.  When  students  critique  each  other’s  work,  the  instructor, 
in  effect,  deputizes  the  members  of  the  class  to  do  the  instruction.  In  this  way,  an 
instructor  can  require  more  written  work  from  the  students  without  increasing  the 
instructor’s  work  load  (Marcoulides  and  Simkin,  1991;  Lynch  and  Golen,  1992; 

Healy  and  Jensen,  1996;  Harp,  1988). 
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Concerns  and  Drawbacks  of  Peer  Feedback 


There  are  several  reasons  why  instructors  are  skeptical  of  the  peer  evaluation 
system.  Several  of  these  concerns  are  less  a  function  of  the  process  of  peer  feedback 
specifically,  and  more  a  function  of  introducing  something  new  to  the  classroom.  Some 
instructors  are  nervous  about  “turning  over  classroom  time”  to  students  (George, 
1984:326).  These  instructors  fear  loss  of  “control”  of  the  classroom  (Liftig,  1990:62). 
Other  instructors  have  tried  peer  feedback  and  had  a  negative  experience,  causing  a 
reluctance  to  try  it  again  (Holt,  1992:384). 

Other  concerns  are  specifically  about  students’  ability  to  help  each  other  through 
the  use  of  feedback.  Concern  is  voiced  that  peer  feedback  would  “just  reinforce  the  same 
errors  they  already  make”  and  that  peer  feedback  would  result  in  a  situation  of  “the  blind 
leading  the  blind”  (Sultan,  1988:65).  There  is  also  concern  that  students  may  not  want  to 
openly  criticize  their  fellow  students’  writing  (Lynch  and  Golen,  1992:48).  Finally,  there 
are  questions  about  whether  or  not  student  reviews  are  consistent  and  effective  and  how 
likely  they  are  to  equal  an  instructor’s  evaluation  of  the  same  paper  (Marcoulides  and 
Simkin,  1991:81). 

The  attitude  of  the  students  is  another  concern.  The  research  suggests  that 

students  will  not  use  the  feedback  gained  from  their  peers  unless  they  think  that  this 

feedback  is  useful  or  valuable.  George  (1985)  states  that  peer  feedback  will  only  improve 

performance  if  the  class  is  convinced  that  it  is  worthwhile.  Also  according  to  George 

(1985:323),  students  are  reluctant  to  use  peer  feedback  if  they  are  “suspicious  of  the 

value.”  Liftig  (1990)  reports  that  students  have  a  tendency  to  ignore  feedback  they  see  as 

critical.  He  states  that  students  do  not  incorporate  this  feedback  into  their  revisions  and 
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are  vocal  in  suggesting  that  these  comments  are  not  important.  Lifitig  (1990)  also  reports 
that  students  who  are  involved  in  his  successful  program  of  peer  feedback  characterize 
peer  feedback  as  useful  in  helping  them  build  their  writing  skills. 

Even  among  teachers  who  want  to  use  peer  feedback,  there  are  concerns  about  the 
students’  ability  to  perform  adequately.  There  is  some  push  to  avoid  peer  feedback  as,  at 
best,  a  neutral  experience  and,  at  worst,  a  negative  experience.  Healy  and  Jensen  (1996) 
caution  that  vaguely-worded  approval  and  sentence  level  comment  need  to  be  avoided. 
Lardner  (1989)  mentions,  as  a  concern,  students’  reluctance  to  give  anything  other  than 
positive  feedback,  probably  because  they  are  afraid  to  hurt  the  author’s  feelings.  Liftig 
(1990:62)  supports  the  use  of  peer  feedback  but  wants  to  avoid  “the  sporadic  anonymous, 
and  sometimes  brutal  commentary  that  had  hurt  feelings  in  previous  classes.” 

Alleviating  Concerns  with  Peer  Feedback 

There  is  consensus  in  the  literature  that  peer  feedback  is  helpful  and  valuable  to 
the  students  (Sultan,  1988;  Hughes  1991;  Liftig,  1990;  Holt,  1992;  Marcoulides,  1991; 
Mangelsdorf,  1992;  MacArthur,  1994;  Katstra,  1987;  Davis,  1996;  Lardner,  1989;  Harp, 
1988),  but  there  is  no  solution  to  the  reluctance  to  add  a  new  technique  to  the  classroom. 
Reluctance  to  give  up  the  “control”  of  the  classroom  is  the  personal  preference  of  the 
instructor,  and  should  not  be  used  as  a  reason  to  avoid  peer  feedback. 

The  solution  to  the  concern  about  students’  ability  to  perform  feedback,  and  their 
attitude  toward  it  is  more  concrete.  One  solution  is  mentioned  repeatedly  in  the  literature: 
simply  teach  students  what  peer  feedback  is  and  how  to  do  it.  Instruction  is 
recommended  as  a  way  to  eliminate  the  vague  positive-only  comments  and  the  brutal 
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feedback  mentioned  by  Liftig  (1990:62),  as  well  as  to  ensure  that  student  reviews  are 
consistent  and  effective. 

Lynch  and  Golen  (1992)  found  that  56  percent  of  teachers  surveyed  do  some  sort 
of  initial  instruction  with  their  students  who  were  going  to  participate  in  peer  feedback. 
Harp  (1988)  says  preparation  is  essential.  Mangelsdorf  asks  students  for  their  feedback 
on  the  peer  review  process.  According  to  the  inputs  she  received,  she  concludes  that 
“peer  review  sessions  have  to  be  carefully  organized”  (Mangelsdorf,  1992:281). 

Elbow  and  Belanoff  (1995)  devote  an  entire  chapter  in  their  text  to  teaching 
students  successful  peer  feedback  techniques.  This  chapter  is  issued  separately  so  that 
teachers  who  do  not  use  their  textbook  will  have  access  to  it.  Holt  (1992)  proposes  a 
technique  for  preparing  students  for  peer  review  that  combines  Elbow  and  Belanoff  s 
techniques  with  Bruffee’s  cumulative  peer-critique  process,  two  techniques  that  she  says 
complement  each  other  well.  According  to  Holt,  Elbow  and  Belanoff  s  techniques  fill  the 
gaps  in  Bruffee’s  system. 

Although  there  is  agreement  on  the  use  of  instruction  to  alleviate  the  concern 
about  student  ability  and  attitude,  authors  do  not  implement  this  solution  the  same  way. 
The  decision  to  teach  students  how  to  do  peer  feedback  is  an  important  first  step  in  the 
peer  feedback  process. 

Implementing  Peer  Feedback 

The  literature  reveals  myriad  ways  to  conduct  peer  feedback.  Many  different 
variables  can  be  assembled  in  a  variety  of  combinations.  These  variables  fall  into  three 
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different  categories:  degree  of  instruction,  procedural  issues,  and  group  issues.  Each 
instructor  hzis  to  decide  what  combination  of  these  variables  to  employ. 

Degree  of  Instruction.  The  one  element  that  is  agreed  upon  by  almost  every  study 
is  that  students  need  some  type  of  instruction  in  how  to  accomplish  peer  feedback  before 
they  can  be  expected  to  provide  it  effectively  (Elbow,  1995;  Harp,  1988;  Lynch,  1992; 
Mangelsdorf,  1992;  Healy,1996;  MacArthur,  1994;  Katstra,  1987;  Lardner,  1989; 

Varone,  1996) .  This  instruction  covers  two  categories.  First,  the  instructor  must  teach 
the  students  how  to  do  peer  feedback.  The  students  must  learn  the  procedures  the 
instructor  expects  them  to  use  when  they  critique  their  fellow  students’  papers.  Then,  the 
instructor  must  teach  the  students  how  to  interact  dming  feedback.  The  students  must 
leam  the  mechanics  of  interaction,  giving  positive  feedback  and  constructive  criticism  as 
opposed  to  attacking  the  author. 

While  there  is  agreement  that  instruction  in  the  peer  feedback  method  is 
necessary,  there  are  many  different  ways  of  accomplishing  it.  But  there  is  one  basic 
decision  every  instructor  must  make:  how  much  time  is  to  be  devoted  to  each  type  of 
instruction.  The  decision  on  duration  of  instruction  is  dictated  by  the  amount  of 
information  the  instructor  wishes  to  provide  and  the  amount  of  time  the  instructor  wishes 
to  devote  to  each  element  to  be  taught.  Depending  on  the  focus,  instruction  can  consist  of 
a  few  minutes  or  an  entire  school  year. 

Every  teacher  also  has  to  choose  their  own  method  for  teaching  these  skills. 
Methods  range  from  informal  techniques  such  as  simply  reminding  students  to  “balance 
positive  and  negative  comments”  and  “look  beyond  the  surface  features  of  the  paper” 
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(Healy,  1996:59)  to  structured  techniques  such  as  modeling  and  student  repetition 
(Mangelsdorf,  1992).  Several  authors  outline  complicated  programs  for  teaching  students 
how  to  accomplish  peer  feedback.  Invariably,  these  authors  spent  more  time  overall  on 
the  instruction  portion  of  the  peer  feedback  experience. 

In  analyzing  the  peer  review  methods  of  78  teachers.  Lynch  (1992)  finds 
“preparatory  interaction  activities”  fall  into  two  main  categories,  large  group  and  small 
group  activities.  The  large  group  activities  include  such  ice-breakers  as  students 
introducing  themselves  to  the  class  and  interviewing  their  classmates.  The  small  group 
activities  include  such  exercises  as  students  going  over  class  material,  discussing  case 
studies  and  actually  critiquing  student  papers.  As  mentioned  above,  56%  of  the  teachers 
he  surveyed  employ  these  “preparatory  interaction  activities”  (Lynch,  1992:46). 

Elbow  and  Belanofif  (1995:4)  make  instruction  in  peer  feedback  methods  the 

subject  of  their  pamphlet  Sharing  and  Responding.  They  believe  “students  can  give  each 

other  remarkably  useful  and  productive  feedback  on  their  writing.  But  most  of  them  need 

substantive  help  and  instruction  in  learning  to  do  so.”  Their  pamphlet  is  a  training 

instrument  for  use  in  teaching  students  how  to  provide  successful  peer  feedback.  As  the 

authors  state,  “we  have  gathered  together  a  full  and  detailed  sequence  of  suggestions  for 

students  to  use  in  sharing  their  writing  with  each  other  and  giving  and  receiving  useful 

responses”  (Elbow  and  Belanoff,  1995:4).  This  program  of  feedback  instruction  begins 

with  easy  tasks  such  as  simple  sharing,  where  students  simply  practice  the  act  of  sharing 

their  work  with  other  students,  then  moves  on  to  more  and  more  complicated  feedback 

techniques.  This  sequence  includes  such  response  techniques  as  “Summary  and 

Sayback,”  where  the  student  actively  summarizes  what  is  read  without  active  criticism, 
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“What  is  almost  said?”  and  “What  do  you  want  to  hear  more  about?”  where  the  student 
responds  to  ideas  that  are  implied  beyond  the  actual  written  words,  and  “Metaphorical 
Descriptions,”  where  the  student  describes  the  writing  in  terms  of  common  everyday 
objects,  such  as  the  weather  or  a  piece  of  clothing,  and  responds  to  the  writing  in  this 
light. 


MacArthur  (1994:25)  says  “Some  form  of  instruction  in  the  cognitive  processes 
involved  in  revision,  whether  through  direct  instruction  or  teacher  conferencing,  appears 
to  be  necessary.”  In  his  classroom,  MacArthur  involves  his  students  in  a  formal  student 
editor  strategy.  This  strategy  has  five  steps.  They  are: 

1 .  LISTEN  and  read  along  as  the  author  reads 

2.  TELL  what  it  was  about  and  what  you  liked  best 

3.  READ  and  make  NOTES.  CLEAR?  Is  anything  hard  to  understand? 

DETAILS?  Where  could  more  details  be  added? 

4.  DISCUSS  your  suggestions  with  the  author. 

5.  Author  makes  revisions  on  the  computer.  (MacArthur,  1994:26) 


These  five  steps  are  taught  to  the  students  using  a  complex  seven-stage  process 
that  teaches  the  unskilled  student  in  increments  imtil  the  student  is  able  to  perform 
successfully  independently.  The  steps  are: 

1 .  Preskill  Development  -  Does  the  student  understand  that  writing  is  a  process 
that  involves  planning,  writing,  revising  and  sharing  his  work? 

2.  Initial  Conference  -  the  teacher  reviews  current  performance  and  discusses  the 
importance  of  revision  with  the  student. 

3.  Discussion  of  Strategy  -  the  teacher  describes  the  strategy  to  the  student  and 
explains  why  it  is  important,  and  when  to  use  it. 

4.  Modeling  of  Strategy  -  The  teacher  goes  through  the  process  of  editing  a  paper 
with  the  student  watching 

5.  Mastery  of  Strategy  -  Students  memorize  the  steps  of  the  strategy 

6.  Collaborative  Practice  -  Students  practice  the  strategy  of  peer  feedback  in 
small  groups  with  the  support  of  the  teacher. 

7.  Independent  Performance  -  Teacher  support  becomes  less  and  less  often  as 
the  students  leam  to  work  in  independent  pairs.  (MacArthur,  1994:26) 
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The  main  goal  of  this  seven-stage  process  is  to  teach  the  students  how,  why,  and 
when  to  use  the  student  editor  strategy.  MacArthur  believes  that  if  the  student 
understands  these  reasons,  the  student  is  more  likely  to  use  the  strategy  after  the 
instmction  period  ends. 

In  the  process  of  organizing  her  peer  review  process,  Mangelsdorf  (1992)  heeds 
her  students’  complaint  that  peer  review  sessions  need  to  be  more  organized  to  be 
successful.  Her  response  to  this  is  to  teach  students  how  to  do  peer  review  before  asking 
them  to  do  it  for  themselves.  Mangelsdorf  s  (1992)  technique  is  less  formal  than  the 
technique  discussed  in  Elbow  and  Belanoff  (1995).  First,  she  models  the  proper  way  to 
do  peer  feedback.  Modeling  is  done  by  critiquing  sample  essays  in  class,  then  letting  the 
students  practice  the  proper  way  to  “make  suggestions  for  revision”  (Mangelsdorf, 
1992:281).  Often  students  are  allowed  to  see  a  peer  feedback  form  completed  by  the 
teacher  as  an  example. 

Healy  and  Jensen  (1996:59)  use  a  relaxed  form  of  instruction  for  teaching  their 
review  board  members  how  to  appropriately  respond  to  their  peer’s  writing.  They  simply 
“remind  them  to  balance  positive  and  negative  comments,”  and  “encourage  them  to  look 
beyond  surface  features  of  a  paper.” 

As  with  every  other  aspect  of  peer  feedback,  there  are  many  different  methods  and 
techniques.  However,  the  authors  all  agree  on  one  point;  students  must  be  trained  in  the 
proper  way  to  accomplish  peer  feedback  if  it  is  to  be  successful. 
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Procedural  Issues.  A  large  part  of  determining  the  amount  and  type  of  instruction 


that  the  students  receive  is  the  number  and  types  of  procedures  that  the  instructor 
introduces.  Several  procedural  issues  should  be  addressed  before  instruction  can  begin. 

The  degree  of  formality  indicates  how  structured  the  feedback  experience  is. 
Several  elements  combine  to  determine  degree  of  formality.  These  elements  are: 
feedback  steps  that  must  be  followed;  feedback  form,  including  type  of  direction;  type  of 
communication  to  be  used  during  the  feedback  effort;  and  the  place  of  the  feedback  effort 
in  the  writing  process. 

Every  instructor  who  practices  peer  feedback  in  the  classroom  introduces  a  set  of 
rules  or  steps  to  the  students.  These  steps  determine  the  structure  of  the  feedback 
sessions  and  vary  in  specificity.  They  range  from  having  the  student  memorize  an  exact 
set  of  steps  that  must  be  followed  each  time  feedback  is  conducted  (MacArthur,  1994),  to 
teaching  students  the  different  techniques  that  are  available  and  allowing  them  to  choose 
how  and  when  to  implement  them  (Elbow  and  Belanoff,  1995). 

Another  element  that  helps  define  degree  of  formality  is  the  use  of  a  peer 
feedback  form.  The  use  of  a  feedback  form  and,  when  a  form  is  used,  the  type  of  form 
also  vary  widely  from  instructor  to  instructor.  Several  researchers  use  no  formal  peer 
feedback  form  (Nelson,  1993,  Sultan,  1988,  Hughes,  1991,  Kowalski,  1989). 

Mangelsdorf  (1992)  recalls  practicing  a  form  of  feedback  where  students  read  each 
other’s  work  in  a  group  setting  and  then  verbally  make  suggestions  for  revision.  Despite 
these  researchers,  use  of  a  formal  peer  feedback  form  appears  extensively  in  the  research. 
The  procurement  and  content  of  the  forms,  as  well  as  the  types  of  answers  expected  on 
the  forms,  differ  according  to  instructor  preference. 
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Lynch  (1992)  observes  that  42.1%  of  teachers  surveyed  who  participated  in  peer 
feedback  used  a  rating  instrument.  He  further  shows  that  of  this  group,  79.5%  develop 
their  own  instrument,  10.3%  take  the  rating  instrument  jfrom  a  published  source,  and 
10.3%  use  a  rating  instrument  that  was  developed  by  the  students  and  teacher  together. 
None  of  the  teachers  siureyed  use  a  rating  instrument  developed  by  students  alone. 

The  most  rigid  of  the  peer  feedback  forms  employs  rating  or  ranking  scales, 
effectively  limiting  the  critiquing  student  to  the  topics  and  responses  the  instructor 
provides.  In  his  study,  Newkirk  (1984)  uses  a  feedback  form  that  asks  participants  to 
both  rate  and  rank  the  fovir  papers  they  are  responsible  for  critiquing.  The  participants 
first  rate  each  individual  paper  on  a  scale  of  1  to  10,  with  10  high  and  0  low;  then  they 
rank  the  papers  relative  to  each  other  on  a  scale  of  1  to  4,  with  1  high  and  4  low. 
Participants  were  not  asked  to  answer  open-ended  questions  about  the  content  of  the 
writing,  but  they  were  asked  to  write  a  paragraph  explaining  the  reason  for  the  rankings 
they  assigned  to  each  paper. 

Still  rigid,  but  more  comprehensive,  are  the  feedback  forms  that  use  a  series  of 
topics  to  be  addressed,  and  likert  scales  for  the  reviewing  student  to  rate  the  papers  on 
those  topics.  Marcoulides  (1991)  makes  use  of  a  detailed  Student  Paper  Evaluation  Form 
that  has  the  peer  evaluator  rate  each  paper  in  ten  specific  areas  in  three  categories  on  a 
three  item  likert  scale,  good/fair/poor.  Figure  1  lists  these  three  categories  and  the  areas 
rmder  them.  While  this  feedback  form  focuses  on  the  use  of  closed  questions,  it  also 
provides  a  section  that  asks  for  conunents  without  providing  a  directive  question. 
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Figure  1.  Marcoulides  (1991)  Evaluation  Form  Categories 
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References 

A  less  rigid  feedback  form  makes  use  of  both  open-ended  questions  and  questions 
that  rely  on  scalar  answers.  Mangelsdorf  (1992)  suggests  that  instructors  develop  this 
type  of  worksheet  for  use  at  more  advanced  levels.  The  worksheet  developed  by 
Mangelsdorf  (1992)  consists  of  specific  questions  requiring  scalar  answers  about  the 
content,  focus,  organization  and  development  of  the  writing.  It  then  asks  open-ended 
questions  that  require  the  student  to  suggest  ways  the  author  could  make  the  essay  better 
in  those  areas.  Mangelsdorf  (1992)  comments  that  these  same  questions  should  not  be 
used  for  every  feedback  session;  rather  questions  should  be  developed  to  coincide  with 
the  instructor’s  focus  at  the  time  the  feedback  form  will  be  used. 

Another  type  of  feedback  form  does  not  use  scalar  answers.  It  allows  the  reviewer 
to  answer  specific  open-ended  questions.  Healy  and  Jensen  (1996)  created  a  feedback 
form  that  focuses  on  having  the  reviewer  make  observations  and  ask  questions  about  the 
content  of  the  writing  as  opposed  to  merely  correcting  the  writing.  Their  intention  is 
twofold.  First,  they  want  to  give  the  student  performing  the  feedback  some  direction 
without  leading  the  student  to  a  particular  answer,  and  second,  they  want  to  discourage 
the  vague  feedback  students  often  give  to  each  other  (e.g.,  “I  really  liked  your  paper”). 
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One  type  of  feedback  form  which  employs  open-ended  questions  is  designed  to 
allow  the  student  doing  the  critiquing  to  answer  broad  questions  in  his  own  words.  Liftig 
(1990:62)  makes  use  of  a  four  question  peer-reaction  worksheet  that  asks  four  specific 
questions  about  the  memorable  words,  phrases,  and  events  in  the  reading  and  is  designed 
to  provoke  thought  about  the  actual  content  of  the  writing  being  critiqued.  The  questions 
are: 


1 .  What  are  the  first  three  words  that  come  to  your  mind  after  reading  this  story? 

2.  What  is  one  memorable  phrase  from  the  story?  Why  do  you  think  it  sticks  in 
your  mind? 

3.  Recall  briefly  a  memorable  scene  from  the  story.  For  what  positive  reasons  do 
you  think  you  remembered  it? 

4.  What  would  be  one  element  of  this  story  that  you,  as  reader,  would  like  to 
read  more  about?  (e.g.,  “I  would  like  to  read  more  about  the  main  character’s 
childhood.”)  (Liftig,  1990:62) 

The  decision  about  the  type  of  feedback  form  to  be  used  does  not  stand  alone. 

The  type  of  communication  to  be  used  during  the  feedback  effort  also  needs  to  be 
selected.  Responses  can  be  either  verbal  or  written  (Hughes,  1991;Sultan,  1988;  Lynch, 
1992),  and  the  type  of  communication  can  be  affected  by  the  use  of  a  feedback  form.  The 
feedback  form  limits  verbal  communication  because  it  ensures  that  at  least  part  of  the 
response  will  be  written.  Commonly,  a  combination  of  verbal  and  written  feedback  is 
used  (Mangelsdorf,  1992).  This  is  a  good  solution  to  the  concern  that  verbal  response 
alone  is  not  as  effective  as  written  response  because  students  have  a  tendency  to  block 
negative  criticism  and  forget  what  is  said  (George,  1985). 

Another  procedural  issue  that  needs  to  be  addressed  is  the  timing  of  the  feedback 
effort.  This  is  important  because  it  helps  to  determine  the  overall  formality  of  the 
feedback  experience.  The  literature  shows  that  the  less  structured  the  feedback  effort,  the 
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more  likely  this  effort  is  to  occur  spontaneously.  In  classrooms  where  the  feedback  effort 
is  used  more  as  a  “lifestyle”  and  less  as  a  task  to  be  accomplished,  the  students  become 
more  comfortable  with  feedback  as  a  whole  and  participate  in  it  often  and  on  an  informal 
basis  (Sultan,  1988;  Kowalski,  1989). 

Last,  three  final  procedures  need  to  be  addressed.  These  procedures  are 
anonymity,  requirement  for  participation,  and  duration  of  the  exercise.  The  research  does 
not  address  the  affect  of  anonymous  vs.  non-anonymous  feedback;  however,  both 
MacArthur  (1994)  and  Liftig  (1990)  require  their  students  to  put  their  name  on  the 
feedback  they  accomplish  so  the  author  knows  who  completed  the  critique.  In  contrast, 
Marcoulides  (1991)  takes  steps  to  preserve  anonymity  when  his  students  participate  in 
their  feedback  sessions.  The  decision  to  conduct  feedback  in  anonymity  is  to  a  certain 
degree  controlled  by  the  communication  method  chosen.  Obviously,  anonymity  can  be 
maintained  only  in  a  situation  where  the  feedback  is  given  and  received  in  written  form. 

The  next  procedure  that  needs  to  be  considered  is  who  will  be  required  to 
participate.  There  are  several  ways  to  do  this,  from  requiring  participation  of  the  entire 
class,  whether  as  a  whole  group  or  broken  down  into  smaller  groups  (Marcoulides,  1991; 
Schramm,  1990;  Mangelsdorf,  1992;  Sultan,  1988;  Hughes,  1991),  to  recruiting 
volmteers,  both  for  providing  writing  that  needs  to  be  reviewed  and  for  reviewing  the 
work(Healy,  1996;  Newkirk,  1984). 

The  final  procedure  that  needs  to  be  considered  is  the  duration  of  the  exercise 
(Lynch,  1 992).  One  group  of  researchers  conducts  peer  feedback  as  a  one-time  event 
(Newkirk,  1984;  Marcoulides,  1995;  Schramm,  1990;  Liftig,  1990).  Students  are  taught 
how  to  do  feedback,  perform  feedback  for  one  paper,  and  are  not  required  to  accomplish 
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it  again.  Other  authors  conduct  peer  feedback  on  a  continual  basis,  usually  for  the 
duration  of  the  school  year  (Sultan,  1988;  Nelson,  1993;  Kowalski,  1989). 

Group  Issues.  Group  issues  cover  another  set  of  variables  that  also  vary  from 
instructor  to  instructor.  In  some  cases  the  group  issues  limit  the  procedures  that  can  be 
used.  These  issues  are  size  of  group  and  composition  of  group. 

Size  of  Group.  The  first  group  decision  the  instructor  has  to  make  is  how 
many  students  to  put  into  each  group.  Essentially,  the  instructor  needs  to  decide  whether 
students  are  going  to  do  peer  feedback  on  a  one-on-one  basis,  in  small  groups,  or  in  large 
groups.  One-on-one  feedback  obviously  limits  the  size  of  the  group  to  two  individuals. 
However,  if  the  decision  is  to  divide  the  class  into  groups  of  more  than  two,  the  instructor 
next  needs  to  determine  how  many  students  to  place  in  each  group. 

Lynch  (1992)  determines  in  his  study  that  13.4  percent  of  respondents  implement 
peer  evaluation  groups  with  two  students,  23.2  percent  of  respondents  include  the  entire 
class,  and  64.2  percent  of  respondents  have  peer  evaluation  groups  of  some  size  in 
between.  Most  of  these  respondents’  peer  evaluation  groups  consist  of  between  three  and 
five  members. 

Composition  of  Group.  The  physical  composition  of  the  group  is  another 
issue  to  be  addressed.  This  issue  consists  of  two  factors,  first,  how  students  are  assigned 
to  each  group,  and  second,  whether  the  group  is  static  or  changing.  While  most 
researchers  group  their  students  randomly  or  do  not  address  this  issue  at  all,  Mangelsdorf 
(1992)  prefers  to  group  her  students  by  ability.  She  notes  that  when  there  is  a  wide 
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variability  between  students’  ability  in  a  feedback  group,  the  better  students  tend  to  give 
good  feedback  to  the  weaker  students  and  receive  little  or  no  feedback  in  return. 

If  the  feedback  exercise  consists  of  more  than  one  feedback  experience,  or  more 
than  one  paper  is  to  be  evaluated,  the  instructor  should  decide  whether  to  use  static 
groups  or  to  require  the  students  to  change  groups  between  exercises.  There  are 
arguments  for  and  against  static  versus  changing  groups.  Sultan  (1988)  keeps  her 
students  in  static  feedback  groups  for  the  duration  of  the  school  year.  Not  only  are  these 
students  assigned  to  the  same  feedback  group,  they  are  seated  at  the  same  table  for  every 
wnting  class  for  the  whole  year.  She  prefers  this  static  group  because  she  notices  a  high 
comfort  level  based  on  familiarity  with  other  team  members.  Even  when  the  students  are 
not  engaged  in  formal  feedback,  they  ask  the  other  students  on  their  feedback  team  for 
assistance  with  their  papers  throughout  the  entire  writing  process. 

Other  authors  have  their  students  change  groups  for  every  exercise  so  that  every 
student  benefits  from  as  many  different  points  of  view  as  possible  (Schramm  and  Rich, 
1990;  Mangelsdorf,  1992).  This  factor  is  especially  important  when  a  teacher  institutes  a 
program  of  one-on-one  peer  feedback.  In  the  case  of  one-on-one  peer  feedback,  the 
students  do  not  have  the  benefit  of  several  different  points  of  view  within  their  own  peer 
feedback  group,  so  pairing  each  student  with  several  different  partners  provides  different 
points  of  view. 

Nelson  (1993)  explores  the  effectiveness  of  verbal  communication  during  group 
feedback  sessions  in  depth.  She  observes  student  groups  participating  in  verbal  feedback 
over  a  period  of  ten  weeks.  She  finds  that  the  effectiveness  of  the  verbal  interaction  has 
to  do  with  the  type  of  interaction  that  occurs.  She  identifies  two  scales  on  which  the 


24 


feedback  experience  can  be  measured.  In  her  study  Nelson  (1993)  finds  that  the  students 
are  either  interactive  or  non-interactive  and  either  cooperative  or  defensive.  Students 
verbally  respond  to  suggestions  made  by  a  peer  (interactive),  or  do  not  (non-interactive). 
If  they  do  respond,  their  response  is  cooperative  (they  are  making  an  effort  to  imderstand 
what  their  team  members  are  telling  them)  or  defensive  (the  writer  displays  behaviors 
such  as  disagreeing  or  justifying).  She  finds  that  students  are  more  likely  to  gain  from 
peer  feedback  groups  if  they  interact  with  the  other  members  of  the  group  in  a 
cooperative  way. 

Computers  and  Peer  Feedback 

As  previously  stated,  much  of  the  literature  suggests  that  peer  feedback  is 
successful  in  helping  students  improve  their  writing.  Considering  the  progress  in 
technology  in  the  last  ten  years,  the  next  logical  step  is  to  use  computers  in  the 
implementation  of  peer  feedback.  This  section  looks  at  the  use  of  computers  during  the 
writing  process.  It  addresses  the  advantages  of  using  computers  during  the  writing 
process  and  peer  feedback,  and  discusses  the  disadvantages  to  computer  use  during  these 
processes. 

Computer  Use  In  Peer  Feedback.  Students  have  been  shown  to  like  writing  with 
computers  better  than  writing  with  paper  and  pencil  (Kowalski,  1990).  Because  of  this,  it 
is  thought  that  peer  editing  might  be  more  effective  if  computers  are  used  (Kowalski, 
1990).  These  two  thoughts  lead  to  the  next  logical  step:  use  of  computers  not  only  during 
the  writing  process,  but  also  during  the  feedback  process.  However,  one  must  heed 
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Mac  Arthur’s  (1994)  caution  that  although  the  computer  is  a  helpful  tool  in  the  feedback 
process,  it  should  not  be  used  as  a  substitute  for  instruction  in  revision. 

There  are  several  suggestions  for  how  to  incorporate  computers  into  the  writing 
and  peer  feedback  processes.  Healy  (1996)  suggests  the  use  of  e-mail  to  supplement  the 
work  of  his  editorial  board.  While  this  suggestion  is  using  computers  to  facilitate  peer 
feedback,  Healy  (1996)  does  not  seem  to  be  suggesting  that  the  use  of  computers  makes 
the  feedback  process  more  effective.  He  is  simply  suggesting  the  use  of  computers  to 
make  the  administrative  tasks  associated  with  the  editorial  board  easier.  MacArthur 
(1 994)  suggests  the  use  of  computers  in  all  stages  of  the  writing  process.  He  reports  the 
use  of  computers  as  especially  useful  during  the  revision  process  (MacArthur,  1994). 

Kowalski  (1990)  outlines  several  different  ways  to  use  computers  during  the 
feedback  process.  She  first  discusses  on  disk  editing,  where  the  students  write  their  paper 
on  the  computer  and  save  it  to  a  disk.  The  peer  editor  then  accomplishes  feedback  by 
inserting  his  comments  right  into  the  text  on  the  disk.  Kowalski  (1990)  then  mentions  a 
colleague  who  uses  computers  for  peer  feedback.  This  colleague  has  his  students  bring  a 
printed  copy  of  their  work  to  class,  and  the  peer  evaluators  use  the  computer  to  fill  out  a 
peer  feedback  form. 

These  authors  agree  that  computers  can  be  advantageous  when  used  during  the 
writing  process;  however,  they  can  also  present  difficulties.  The  next  two  sections 
address  these  advantages  and  disadvantages. 

Advantages  of  Computer  Use .  The  literature  outlines  several  advantages  of 
computer  use  over  the  standard  pencil  and  paper  way  of  writing  and  giving  feedback. 
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MacArthur  (1994)  is  not  alone  in  his  praise  of  the  use  of  the  computer  to  “ease  the 
process  of  revision.”  Kowalski  (1990)  also  mentions  the  relative  ease  of  revision  when  it 
is  done  on  the  computer  screen.  These  comments  focus  on  the  elimination  of  the  mess 
involved  in  making  comments  and  changes  on  paper  with  a  pencil.  MacArthur  (1994) 
also  comments  on  the  use  of  computers  to  speed  up  the  writing  process,  especially  for 
students  who  have  trouble  with  the  physical  act  of  writing  with  a  pencil  (MacArthur, 
1994).  Another  practical  advantage  of  using  computers  to  accomplish  peer  feedback  is 
the  elimination  of  paper  copies.  Students  no  longer  have  to  generate  several  copies  of 
their  paper;  one  electronic  copy  is  sufficient  (Kowalski,  1990). 

Disadvantages  of  Computer  Use.  In  Kowalski’s  (1990:38)  classroom  these 
positive  aspects  are  overshadowed  by  the  negative  aspects  discovered  by  her  students. 
Kowalski  (1990:38)  reports  that  her  students  were  “quite  vocal  about  expressing  their 
dissatisfaction”  regarding  the  following  point:  there  was  concern  that  the  computer  was 
being  used  for  peer  editing  because  it  was  there,  and  not  because  that  was  the  best  way  to 
accomplish  the  task.  More  importantly,  her  students  complained  that  they  had  trouble 
reading  entries  of  any  length  on  the  computer  screen. 

Other  problems  that  impact  the  use  of  the  computer  for  peer  editing  are  physical 
problems.  These  problems  include  slow  equipment,  time  consumed  with  booting  up  the 
system,  and  time  spent  trading  disks  (Kowalski,  1990).  Time  spent  finding  a  system  for 
the  student  to  use  outside  of  class  time  was  also  a  concern  (Kowalski,  1989). 

Kowalski’s  (1990)  final  conclusion  was  that  peer  editing  xising  the  computer  was 
not  the  positive  experience  she  had  hoped  it  would  be.  She  states  that  her  students  started 
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the  school  year  motivated  to  write  and  edit  on  disk.  However,  the  problems  encountered 
using  the  computer  for  feedback  soon  negatively  impacted  their  motivation. 

Chapter  Summary 

In  this  chapter  writing  as  a  process  has  been  discussed  and  the  concept  of  peer 
feedback  and  ways  to  implement  it  have  been  introduced.  The  reader  should  also 
understand  the  drawbacks  and  concerns  of  using  peer  feedback  in  the  classroom  and  the 
use  of  computers  in  the  peer  feedback  process.  The  stage  has  been  set  to  discuss  the 
particular  feedback  methods  examined  in  this  study.  Chapter  III  introduces 
implementation  of  peer  feedback  in  the  English  Department  at  the  United  States  Air 
Force  Academy,  the  collection  of  the  data  for  this  study,  and  the  method  used  to  analyze 
the  data  and  answer  the  research  questions. 
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III.  Methodology 


Introduction 

This  chapter  presents  the  information  required  to  imderstand  this  research 
endeavor.  It  outlines  the  background  information  that  shapes  the  study.  This  information 
includes  a)  the  original  LAN  session  procedures  instituted  in  the  Academy  English 
Department,  b)  the  procedures  instituted  for  online  peer  feedback,  c)  the  procedures 
instituted  for  face-to-face  feedback,  and  d)  the  procedures  followed  for  students  who 
receive  no  feedback.  Next,  this  chapter  discusses  the  steps  followed  to  collect  the  data, 
including  a  discussion  of  the  questionnaire  used,  and  the  method  of  collection.  Finally, 
this  chapter  discusses  the  methods  used  to  analyze  the  data  collected. 

Background 

To  design  this  study,  the  researcher  first  examined  how  the  Academy  English 
Department  conducts  peer  feedback.  At  the  beginning  of  the  1995-1996  school  year,  the 
English  Department  implemented  an  online  peer  feedback  program  using  the  available 
technology.  This  available  technology  consists  of  a  computer  on  each  teacher’s  desk  and 
in  each  student’s  dormitory  room.  These  computers  are  hooked  into  the  Academy’s 
LAN.  The  sessions  in  this  program  are  commonly  called  “LAN  sessions”  by  the  students 
and  faculty. 

Instructors  were  encoiiraged,  but  not  required,  to  participate  in  this  program  as  it 
is  written.  As  a  result,  some  instructors  used  the  program  in  its  entirety,  some  instructors 
used  the  program  with  minor  changes,  and  some  instructors  used  only  parts  of  the 
original  LAN  session  procedures. 
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LAN  Session  Procedures.  The  peer  feedback  program  is  intended  to  be  integrated 
into  the  normal  semester-long  Freshman  English  class  (English  111).  This  means  that 
writing  sessions  and  peer  feedback  sessions  are  accomplished  during  assigned  class  time. 
This  is  in  addition  to  the  usual  activities  of  reading  assigned  essays  for  homework  and 
discussing  the  essays  in  class.  Peer  feedback  sessions  are  intended  mainly  to  help  the 
students  become  better  writers,  not  as  an  exercise  on  which  to  base  the  final  grade.  Only 
ten  percent  of  the  course  grade  is  based  on  the  student’s  “LAN  idea  sessions”  folder. 

This  folder  contains  the  results  of  all  LAN  sessions  for  the  semester.  The  remaining 
ninety  percent  of  the  course  grade  is  based  on  class  participation  and  three  papers  the 
students  are  required  to  write  independently  of  the  LAN  sessions. 

The  English  Department’s  first  step  in  instituting  a  program  of  peer  feedback  is  to 
teach  the  students  how  to  provide  feedback  that  is  helpful  (versus  harmful  or  neutral)  to 
each  other.  This  is  done  during  the  first  two  weeks  of  the  semester  using  Elbow  and 
Belanoff  s  (1995)  text,  Sharing  and  Responding.  Students  are  required  to  read  the  text 
and  to  be  prepared  to  discuss  the  techniques  covered  during  class.  This  feedback  training 
introduces  the  idea  of  peer  feedback  to  the  students,  teaches  them  the  techniques  to  use 
when  accomplishing  peer  feedback,  and  makes  them  comfortable  with  the  process  of  peer 
feedback.  The  main  goal  is  to  ensure  that  the  feedback  experience  is  positive  for  everyone 
involved. 

After  this  material  is  discussed  in  class  and  the  instructor  feels  confident  that  the 
students  imderstand  the  concepts  well  enough  to  participate  positively  in  peer  feedback, 
the  LAN  sessions  begin.  Although  LAN  sessions  are  accomplished  during  regularly 
scheduled  class  time,  unlike  regularly  scheduled  classes  where  the  students  report  to  the 
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classroom,  students  do  not  report  to  the  class,  but  instead  participate  in  LAN  sessions 
from  their  dormitory  rooms.  Each  student’s  room  contains  a  computer  that  is  hooked  up 
to  the  Local  Area  Network.  LAN  sessions  are  dependent  on  this  equipment.  Each  LAN 
session  is  divided  into  two  parts,  part  A  and  part  B.  These  parts  are  accomplished  dtiring 
two  consecutive  class  periods. 

In  part  A,  the  students  use  the  computers  in  their  room  during  the  regularly 
scheduled  class  time.  First,  they  check  their  electronic  mail  (e-mail)  for  a  message  from 
the  instructor.  This  message  contains  the  direction  the  students  need  to  write  the  essay 
for  that  particular  LAN  session.  The  direction  is  in  the  form  of  questions  about  the 
assigned  reading  that  are  to  be  answered  in  the  essay.  Students  are  given  the  entire  50 
minute  class  period  to  complete  their  essay  online.  If,  at  any  time,  a  student  has  a 
question  about  the  assignment,  the  instructor  is  online  and  will  receive  and  respond  to  e- 
mail  messages  from  the  student.  Once  the  essays  are  complete,  the  students  post  them  to 
the  appropriate  class  folder  on  the  common  drive  so  that  they  may  be  accessed  by  the 
instructor  and  other  students  in  their  class. 

Part  B  is  accomplished  during  the  next  class  period.  In  this  part,  the  students 
access  the  essay  of  the  student  assigned  to  them.  This  essay  was  posted  on  the  common 
drive  the  previous  class  period.  The  students  critique  the  essays  online,  and  post  them, 
with  comments,  on  the  common  drive.  These  critiques  can  be  either  in  the  form  of 
comments  inserted  into  the  text  of  the  essay,  comments  typed  at  the  end  of  the  essay,  or 
comments  posted  as  annotations  to  the  text.  After  part  B  of  the  LAN  session,  students  are 
responsible  for  reviewing  the  feedback  they  receive  on  their  essay.  The  decision  to  revise 
the  essay  based  on  the  peer  feedback  comments  received  is  left  up  to  the  individual 
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student;  however,  guidance  from  the  course  outline  is  as  follows:  “To  be  considered  for 
an  above-average  grade,  your  final  folder  must  include  all  five  LAN  sessions  and  either 
one  major  revision  or  two  minor  revisions.” 

Although  this  standard  online  peer  feedback  procedure  exists,  not  all  instmctors 
follow  this  procedure.  As  a  result,  three  different  populations  of  students  exist;  students 
who  participate  in  parts  A  and  B  of  the  LAN  sessions  and  receive  peer  feedback  online, 
students  who  participate  in  part  A  of  the  LAN  sessions  and,  instead  of  participating  in 
part  B,  receive  peer  feedback  face-to-face  during  class  time,  and  students  who  participate 
in  part  A  of  the  LAN  sessions  and  do  not  receive  peer  feedback. 

Online  Feedback  Procedures.  Two  instructors  implement  online  peer  feedback  in 
their  classrooms  exactly  as  outlined  above.  However,  assignment  of  feedback  partners  is 
accomplished  differently  for  each  instructor.  One  instructor  randomly  assigns  two  or 
three  students  to  each  essay,  so  each  author  receives  comments  online  from  two  or  three 
classmates.  This  instructor  does  not  assign  the  same  feedback  partners  to  each  individual 
from  LAN  session  to  LAN  session.  The  intent  is  for  each  student  in  the  class  to  critique 
the  writing  of  each  other  student  in  the  class  at  least  once. 

The  other  instructor  assigns  students  in  pairs;  each  student  critiques  one  paper  and 
each  author  receives  feedback  from  the  same  student  whose  paper  he  critiques.  The 
instmctor  assigns  these  partners  based  on  similarities.  He  assigns  students  to  provide 
feedback  to  other  students  who  have  written  essays  on  the  same  topic  or  who  have  made 
similar  mistakes  in  their  writing.  For  the  purpose  of  this  study,  the  46  students  of  these 
two  instructors  form  the  “online  feedback”  group. 
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Face-to-face  Feedback  Procedures.  Two  instructors’  students  participate  in  face- 


to-face  feedback.  These  students  participate  in  part  A  of  the  online  peer  feedback 
sessions  and  do  not  participate  in  part  B.  Instead,  these  students  are  required  to  report  to 
the  classroom  with  a  hard  copy  of  the  essay  they  wrote  during  part  A  for  face-to-face  peer 
feedback. 

Once  the  students  report  to  the  classroom,  they  are  given  a  single  feedback 
partner.  Sometimes  the  students  are  assigned  a  partner  randomly,  sometimes  the  students 
are  assigned  parmers  by  the  instructor  based  on  skill  level,  and  sometimes  the  students 
are  allowed  to  select  their  own  feedback  partner.  Students  then  exchange  copies  of  the 
essays  they  had  written  during  part  A  of  the  LAN  session  and  are  given  twenty  to  thirty 
minutes  to  make  comments  directly  on  the  hard  copy  of  their  partner’s  essay.  After  the 
allotted  time  has  elapsed,  students  give  the  essay  with  their  comments  back  to  their 
partner  and  verbally  explain  the  comments  they  had  made  on  the  paper.  These  50 
students  form  the  “face-to-face  feedback”  group. 

No  Feedback  Procedures.  One  instructor  does  not  have  his  students  participate  in 
peer  feedback  at  all.  For  part  A  of  the  LAN  sessions  these  students  report  to  their 
dormitory  rooms  and  write  essays  in  answer  to  questions  about  the  reading.  For  part  B  of 
the  LAN  sessions,  this  instructor  has  his  students  report  to  the  classroom.  There,  as  a 
class,  they  discuss  the  questions  asked  in  the  e-mail.  These  33  students  form  the  “no 
feedback”  group. 
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Research  Questions 


The  overarching  question  is:  “Is  the  online  peer  feedback  program  effective?”  To 
answer  this  question,  three  research  questions  need  to  be  examined.  The  research 
questions  and  the  hypotheses  tested  to  answer  them  are  as  follows: 

Research  Question  1:  Will  students  who  received  peer  feedback,  either  online  or 
face-to-face,  report  more  improvement  in  writing  skills  than  students  who  did  not 
participate  in  feedback? 

Hypothesis  la:  Students  in  all  three  groups  will  report  a  significant 
change  in  their  writing  skills  between  the  beginning  of  the  semester  and 
the  end  of  the  semester. 

Hypothesis  lb:  Students  in  all  three  groups  will  report  improvement  in 
their  writing  skills. 

Research  Question  2:  Will  students  who  participate  in  online  peer  feedback 
exhibit  a  stronger  belief  that  LAN  sessions  make  writing  more  enjoyable  than 
students  who  participate  in  face-to-face  feedback? 

Hypothesis  2:  Both  the  online  feedback  and face-to-face  feedback  groups 
will  agree  with  the  statement  “LAN  sessions  make  writing  more 
enjoyable.  ” 

Research  Question  3:  Will  students  who  participate  in  peer  feedback  report  that 
feedback  is  worthwhile? 

Hypothesis  3:  The  mean  score  for  worth  will  be  higher  than  the 
midpoint. 
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Data  Collection 


Before  the  advent  of  the  peer  feedback  system,  students  were  required  to  write 
three  8-  to  10-page  papers.  With  the  start  of  the  peer  feedback  system,  students  are 
required  to  write  five  short  essays  online  in  addition  to  the  three  8-  to  10-page  papers. 
Since  the  students  are  writing  more  now  than  was  required  of  them  in  previous  years, 
analysis  of  grade  data  collected  over  a  period  of  years  will  not  distinguish  whether  peer 
feedback,  the  increased  amount  of  writing,  or  the  interaction  of  the  two  is  responsible  for 
any  improvement  in  the  grades.  As  a  result,  this  research  is  based  on  self-reported 
perceptual  data. 

Classes  were  chosen  for  participation  based  on  the  type  of  feedback  they  received. 
Students  were  randomly  assigned  to  instructors  at  the  beginning  of  the  academic  year  and 
each  instructor  chose  his  method  of  implementing  peer  feedback. 

Questionnaire.  Students  involved  in  the  study  were  asked  to  respond  to  a 
questionnaire  which  explored  their  experiences,  both  with  peer  feedback  and  with  writing 
in  general.  For  purposes  of  this  study,  there  were  two  different  questionnaires,  one  for 
students  who  were  members  of  the  online  feedback  and  face-to-face  feedback  groups,  and 
one  for  students  who  were  members  of  the  no  feedback  group. 

Students  in  the  online  and  face-to-face  feedback  groups  answered  the 
questionnaire  containing  eleven  questions  contained  in  appendix  A.  This  questionnaire 
was  divided  into  four  sections.  The  first  section  consisted  of  two  open-ended  questions 
that  dealt  with  the  type  of  feedback  students  receive  or  expect  to  receive  from  their  peer 
feedback  partners. 
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Liftig  (1990)  examined  his  students  comments  about  the  feedback  they  received 
and  discovered  that  his  students  preferred  specific  feedback  over  vague  feedback. 
Mangelsdorf  (1 992)  also  explored  the  types  of  feedback  received  by  her  students.  She 
solicited  responses  from  her  students  and  placed  the  responses  to  these  questions  into  one 
of  three  categories;  positive,  negative,  or  mixed  (positive  and  negative).  She  found  that  a 
majority  of  her  students  reported  positive  comments.  Questions  in  section  one  of  the 
questionnaire  were  taken  directly  from  the  study  conducted  by  Mangelsdorf  (1992). 

The  second  section  of  the  questionnaire  consisted  of  four  questions  that  dealt  with 
the  students’  opinions  concerning  the  worth  of  feedback.  The  literature  showed  that 
successful  peer  feedback  resulted  in  improved  writing  skills.  This  improvement  came 
fi-om  a  two-stage  process.  In  the  first  stage  the  student  accepted  the  feedback.  For  this  to 
happen,  the  student  had  to  believe  that  peer  feedback  is  useful  and  valuable  (George, 
1985;  Mangelsdorf,  1992).  If  the  student  accepted  the  feedback  in  stage  one,  progression 
to  stage  two  occurred:  incorporation  of  the  suggestions  into  his  writing.  This  inclusion  of 
the  ideas  resulted  in  improved  writing  skills.  The  measure  of  the  students’  attitude  about 
these  two  intermediate  steps  and  their  attitude  about  peer  feedback  improving  their 
writing  skills  were  indications  of  whether  or  not  the  student  thinks  that  the  feedback 
experience  is  worthwhile.  The  items  on  the  questionnaire  that  addressed  usefulness  and 
value  were  also  taken  directly  firom  Mangelsdorf  s  (1992)  study. 

The  Air  Force  Academy  is  a  competitive  environment.  The  ultimate  indication  of 
the  student’s  place  in  this  competition  is  the  grades  received.  English  Department  faculty 
members  indicate  that  most  students’  primary  concern  is  their  grades.  This  concern  leads 
to  student  inquiry  as  to  what  can  be  done  to  improve  their  grades.  Because  of  the 
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importance  that  the  students  place  on  their  grades,  it  is  thought  that  the  belief  that  peer 
feedback  will  result  in  a  better  grade  will  be  an  indication  of  the  student’s  belief  that 
feedback  is  worthwhile. 

As  outlined  by  the  above  paragraphs,  the  concept  of  worth  is  indicated  by  student 
beliefs  about  whether  the  feedback:  is  usefiil,  is  valuable,  is  likely  to  make  their  writing 
improve,  and  is  likely  to  help  them  make  a  better  grade.  Questions  3  through  6  on  the 
questionnaire  address  the  different  dimensions  of  worth.  Students  were  asked  to  rate  then- 
degree  of  agreement  to  the  statements  on  the  five-point  likert  scale  found  Figure  2. 


Figure  2.  Five  Point  Likert  Scale 
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These  four  items  combined  to  determine  a  score  for  “worth”  and  are  listed  below: 

Q3.  “I  find  it  useful  to  have  my  classmates  read  my  papers  and  give  me 

feedback.” 

Q4.  “My  writing  is  likely  to  improve  because  of  peer  feedback.” 

Q5.  “The  feedback  my  classmates  provide  is  valuable  to  me.” 

Q6.  “Receiving  peer  feedback  helped  me  make  a  better  grade.” 

To  assess  whether  grouping  these  items  represents  a  reliable  means  for  measuring 
the  construct  of  worth,  a  reliability  analysis  was  conducted  to  measure  internal 
consistency.  The  internal  consistency  measure  for  items  3  through  6  yielded  a 
Cronbach’s  alpha  of  .86.  This  suggests  that  all  four  questions  relate  to  each  other 
strongly  and  therefore  are  all  indicators  of  the  concept  of  worth. 
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The  third  section  asked  the  students  to  report  change,  or  improvement  in  their 
writing  skills  over  the  course  of  the  semester.  This  was  accomplished  by  asking  two 
questions.  Question  seven  asked  the  students  to  rate,  on  the  scale  in  Figure  3,  their 
writing  skills  at  the  beginning  of  the  semester.  Question  eight  asked  the  students  to  rate, 
on  the  same  scale,  their  writing  skill  when  the  questionnaire  was  administered. 


Figure  3.  Ten  Point  Likert  Scale 
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The  fourth  section  dealt  with  the  students’  opinion  of  whether  or  not  LAN 
sessions  make  writing  more  enjoyable.  This  section  was  added  at  the  request  of  the 
sponsor. 

Students  who  received  no  feedback  answered  the  four  questions  which  composed 
part  three  of  the  questionnaire  described  previously.  These  four  questions  were 
renumbered  when  administered  to  this  group.  Appendix  B  contains  the  questionnaire 
administered  to  the  students  in  the  no  feedback  group. 

Questionnaires  were  administered  to  and  completed  by  the  students  during  part  B 
of  the  fifth  and  final  LAN  session  of  the  semester.  This  LAN  session  was  scheduled 
approximately  two  weeks  before  the  end  of  the  semester. 

Questionnaires  were  disseminated  in  two  different  ways.  The  online  feedback 
group  received  online  questionnaires  and  were  instructed  to  answer  these  questionnaires 
in  the  same  manner  they  accomplish  feedback.  They  were  to  respond  to  the  questions 
online  to  post  the  completed  questionnaire  to  the  common  drive.  The  instructors  of  this 
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group  then  copied  all  the  questionnaires  to  a  disk.  Students  in  the  face-to-face  and  no 
feedback  groups  filled  out  a  paper  copy  of  the  questionnaire  during  class  time. 


Data  Analysis 

The  responses  to  the  questions  on  the  questionnaire  are  divided  into  four  smaller 
sets.  The  data  sets  the  questions  belong  to,  the  concept  they  measure,  the  research 
question  they  relate  to,  and  the  feedback  groups  which  were  canvassed  for  information 
are  shown  in  Table  1 .  The  method  used  in  analyzing  these  data  sets  is  outlined  below. 


Table  1.  Research  Questions  as  They  Relate  to  Data  Sets 


Research 

Question 

Data  Set 

Concept 

Measured 

Questions 

Online 

Feedback 

Face  to  Face 

Feedback 

No  Feedback 

1 

One 

mprovemen 

7,8 

yes 

yes 

yes 

2 

Two 

Enjoyment 

11 

yes 

yes 

no 

3 

Three 

Worth 

Em 

yes 

yes 

no 

n/a 

Four 

Types  of 
Feedback 

1,2 

yes 

yes 

no 

Data  Set  One.  Questions  seven  and  eight  ask  the  students  to  rate,  on  the  scale  in 
Figure  3,  their  writing  skills  at  the  begiiming  and  end  of  the  semester,  respectively.  The 
data  was  then  separated  by  treatment  group  and  a  paired  sample  T-test  was  accomplished 
to  determine  if  there  if  there  was  a  significant  difference  between  the  before  and  after 
scores. 

An  improvement  score  was  derived  from  the  data  by  subtracting  the  score  for 
question  seven  from  the  score  for  question  eight.  The  data  was  then  examined  for 
anomalous  responses.  Once  anomalous  data  was  removed,  the  means  of  the  improvement 
scores  were  compared  using  a  one-way  ANOVA. 
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Data  Set  Two.  Question  1 1  was  scored  on  the  scale  in  Figure  2.  The  results  were 


then  divided  into  two  groups,  online  feedback  and  face-to-face  feedback.  Finally,  the 
data  was  analyzed  using  visual  comparison  of  the  means  and  an  independent  samples  T- 
test. 


Data  Set  Three.  The  questions  used  to  determine  worth  were  answered  using  the 
scale  Figure  2.  Once  the  responses  were  collected  they  were  separated  into  two  groups, 
online  feedback  and  face-to-face  feedback.  The  means  of  each  group  were  compared  for 
each  item  in  the  set  to  determine  if  they  were  above  the  midpoint.  A  comparison  of 
means  between  the  groups  was  then  accomplished  using  a  one-way  ANOVA  to  determine 
if  there  was  a  significant  difference  between  the  means  for  any  individual  question.  The 
scores  were  then  aggregated  and  an  overall  mean  score  for  worth  was  determined  for  both 
the  online  and  face-to-face  groups.  A  comparison  of  these  means  was  then  accomplished 
using  an  independent  samples  t-test. 

Data  Set  Four.  The  two  questions  that  comprise  data  set  four  were  open-ended 
questions  that  were  not  conducive  to  empirical  analysis.  With  the  consensus  of  another 
researcher,  a  list  of  responses  was  created,  then  these  responses  were  assigned  to  three 
broad  categories;  content,  flow,  and  grammar.  The  responses  were  divided  into  online 
feedback  and  face-to-face  feedback.  The  percentage  of  responses  for  each  question  that 
fell  into  each  category  were  compared  in  total  and  between  groups  to  see  if  there  was  any 
difference  between  the  responses  for  the  two  questions. 
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Summary 

This  chapter  has  outlined  the  information  necessary  for  the  completion  of  this 
study.  Once  the  necessary  background  information  was  explained,  research  questions  to 
be  answered  were  identified,  methods  used  for  data  collection  were  described,  and  the 
methods  that  will  be  used  to  analyze  the  data  once  it  was  collected  were  profiled.  Now 
that  an  understanding  of  the  methodology  for  data  collection  exists,  Chapter  IV  presents 


the  results  of  that  analysis. 


IV.  Findings  and  Analysis 


Overview 

This  chapter  shows  the  results  and  analysis  of  the  data  collected.  First,  the  data 
used  to  test  each  hypothesis  is  presented.  Then,  the  data  collected  to  support  the 
discussion  on  types  of  feedback  received  and  expected  is  analyzed. 

Results 

Data  from  one  instructor  was  eliminated  for  two  reasons;  first,  the  students  in  the 
class  were  not  all  randomly  assigned  to  that  class  (the  class  contained  a  high  number  of 
remedial  students),  and  second,  the  instructor  practiced  a  combination  of  face-to-face  and 
online  peer  feedback.  This  mixture  of  feedback  methods  precludes  their  membership  in 
any  one  of  the  three  groups  tested. 

Scores  for  writing  skill  from  two  individual  students  (one  in  the  online  feedback 
group  and  one  in  the  no  feedback  group)  resulted  in  negative  improvement  scores.  A 
negative  improvement  score  was  considered  invalid.  Since  these  scores  were  derived 
from  self  reported  perceptual  data,  there  is  a  possibility  that  a  negative  improvement 
score  could  be  the  result  of  some  factor  other  than  deterioration  in  writing  skills. 

Research  Question  1 .  Research  Question  1  asks  if  students  who  received  peer 
feedback,  either  online  or  face-to-face,  will  report  more  improvement  in  writing  skills 
than  students  who  did  not  participate  in  feedback.  The  two  hypotheses  used  to  test  this 
question  and  the  results  of  the  data  analysis  are  below. 
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Hypothesis  la:  Students  in  all  three  groups  will  report  a  significant  change  in 
their  writing  skills  between  the  beginning  of  the  semester  and  the  end  of  the 
semester. 

As  expected,  all  three  groups  (online,  face-to-face,  none)  report  significant 
difference  (p<=.001)  in  their  writing  skills  from  the  beginning  of  the  semester  to  the  end 
of  the  semester  as  shown  in  Table  2. 


Table  2.  Comparison  of  Means  for  Improvement  Scores:  Within  Group 


Group 

N 

mean  writing 
score 

t 

P 

Online 

46 

before 

after 

5.8200 

7.4200 

10.563 

p<=.001 

Face  to  Face 

50 

before 

after 

5.6061 

6.9091 

7.160 

p<=.001 

None 

33 

before 

after 

5.3696 

7.3696 

12.000 

p<=.001 

Hypothesis  lb:  Students  in  all  three  groups  will  report  improvement  in  their 
writing  skills 

All  three  groups  report  a  positive  improvement  score,  indicating  that  they  perceive 
improvement  in  their  writing  skills.  These  scores  are  shown  in  Table  3. 

Table  3.  Comparison  of  Means  for  Improvement:  Between  Groups 


Group 

(Mean 

Improvement) 

Online 

Face  to  Face 

Online 

(2.000) 

Face  to  Face 
(1.608) 

F=2.030 

p=.085 

F=.617 

p=.688 

F=  1.323 
p=.284 

There  is  no  significant  difference  between  the  improvement  scores  of  the  three 
groups  at  the  p=.05  level,  also  shown  in  Table  3,  suggesting  that  while  the  members  of  all 
three  treatment  groups  reported  significant  improvement  in  their  writing  skills  from  the 
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beginning  of  the  semester  to  the  end  of  the  semester,  no  group  reported  significantly  more 
improvement  than  any  other. 

Research  Question  2.  Research  Question  2  asks  if  students  who  participate  in 
online  peer  feedback  will  exhibit  a  stronger  belief  that  LAN  sessions  make  writing  more 
enjoyable  than  students  who  participate  in  face-to-face  feedback.  The  hypothesis  tested 
for  this  research  question  and  the  results  of  the  data  analysis  are  below. 

Hypothesis  2:  Both  the  online  feedback  and  face-to-face  feedback  groups  will 

agree  with  the  statement  “LAN  sessions  make  writing  more  enjoyable.” 

The  means  shown  in  Table  4  suggest  agreement  with  the  statement  on  the  part  of 
the  students  who  receive  peer  feedback  online,  and  possible  neutral  feelings  with  the 
statement  on  the  part  of  the  students  who  receive  feedback  face-to-face.  Because  the 
mean  of  the  face-to-face  group  is  close  to  the  midpoint  of  the  scale,  this  score  may 
indicate  neutrality  to  the  statement  rather  than  disagreement.  Because  the  mean  of  the 
online  group  is  close  to  the  score  for  agree  on  the  scale,  this  score  may  indicate  agreement 
with  the  statement. 


Table  4.  Comparison  of  Means  for  Enjoyment 


Group 

Mean 

t 

P 

Online 

3.717 

3.851 

p<=.001 

Face  to  Face 

2.932 

Also  as  shown  in  Table  4,  the  online  and  face-to-face  feedback  groups  showed  a 
significant  difference  in  the  mean  answer  to  this  question,  suggesting  that  the  online 
group  members  agreed  more  strongly  with  the  statement  than  the  face-to-face  group 
members. 
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Research  Question  3.  Research  Question  3  asks  if  students  who  participate  in 
peer  feedback  report  that  feedback  is  worthwhile.  As  outlined  in  Chapter  III,  questions  3 
through  6  address  the  four  dimensions  of  worth:  students’  beliefs  about  whether  the 
feedback  is  useful,  is  valuable,  is  likely  to  make  their  writing  improve  and,  is  likely  to 
help  them  make  a  better  grade. 

Hypothesis  3:  The  mean  score  for  worth  will  be  higher  than  the  midpoint. 

The  mean  scores  for  worth  for  each  type  of  feedback  are  in  Table  5.  Both  means 
are  above  the  midpoint  of  three,  suggestion  that  both  groups  believe  that  feedback  is 
worthwhile.  An  independent  samples  t-test  shows  that  the  difference  between  the  means 
for  the  online  and  face-to-face  groups  is  significantly  different  at  the  p=.02  level.  This 
suggests  that  the  face-to-face  feedback  group  believes  that  feedback  is  more  worthwhile 
than  the  online  feedback  group. 


Table  5.  Comparison  of  Means  for  Worth 


Mean 

t 

P 

Online 

3.78 

-2.401 

p=.017 

Face  to  Face 

3.96 

The  means  for  each  question  by  type  of  feedback  are  in  Table  6.  All  of  the  means 
are  above  the  midpoint  of  three,  suggesting  that  both  groups  agree  with  all  four 
statements.  This  woiild  also  suggest  that  both  groups  find  feedback  to  be  worthwhile. 

An  independent  samples  t-test  was  conducted  to  determine  if  the  groups  reported 
significantly  different  levels  of  agreement  for  any  of  the  individual  question.  At  p=.05 
level  of  significance,  the  online  and  face-to-face  feedback  groups  had  only  a  significantly 
different  mean  for  question  6:  “Receiving  Peer  feedback  helped  me  make  a  better  grade,” 
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suggesting  that  while  both  the  online  and  face-to-face  feedback  groups  agree  that 
feedback  helped  them  make  a  better  grade,  the  face-to-face  feedback  group  agreed  more 
strongly. 


Table  6.  Comparison  of  Means  for  Questions  that  Measure  Worth 


Question 

Online 

Mean 

Face  to  Face 
Mean 

t 

P 

3 

4.022 

4.157 

-0.964 

0.338 

4 

3.878 

3.941 

-0.384 

0.702 

5 

3.889 

4.020 

-1.014 

0.313 

6 

3.300 

3.725 

-2.594 

0.011 

Types  of  Feedback  Received.  The  original  intent  of  collecting  the  data  in  data  set 
four  was  to  explore  the  types  of  feedback  the  students  received  and  the  types  of  feedback 
the  students  formd  most  helpful  from  the  point  of  view  of  the  two  scales  found  in  the 
literature:  positive/negative  (Mangelsdorf,  1992)  and  vague/specific  (Liftig,  1990).  The 
data  collected  proved  to  be  incompatible  with  categorization  into  these  categories. 

Several  different  researchers  voiced  the  belief  that  peer  feedback  is  intended  to  improve 
writing,  not  by  focusing  on  mechanics  and  flow,  but  by  focusing  on  content  (Sultan, 

1988;  Liftig,  1990;  MacArthur,  1994).  It  was  also  reported  that  students  find  feedback 
that  addresses  content  more  helpful  than  feedback  that  addresses  mechanics  or  flow 
(Newkirk,  1984).  It  is  expected  that  both  feedback  groups  will  report  that  the  majority  of 
the  comments  they  received  will  be  related  to  content.  It  is  also  expected  that  both 
groups  will  report  a  preference  for  feedback  related  to  content.  The  data  collected  for  the 
first  two  questions  were  placed  in  the  broad  categories  content,  flow,  and  mechanics. 
Comments  were  scored  in  the  “content”  category  if  they  mentioned  feedback  that 
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improved  the  content,  pointed  out  the  point  of  view  of  the  paper,  or  gave  ideas  for 
increasing  the  depth  of  the  analysis  of  the  reading.  Comments  were  scored  in  the  “flow” 
category  if  they  mentioned  feedback  that  helped  improve  transitions,  clarify  thesis 
statements,  and  organize  thoughts  better.  Comments  were  scored  in  the  “mechanics” 
category  if  they  mentioned  feedback  that  focused  on  punctuation,  spelling  and  grammar. 
There  is  one  caveat:  scoring  the  data  this  way  was  somewhat  subjective.  Since  the 
questions  were  open-ended,  no  two  responses  were  alike,  and  while  several  concepts 
were  repeated,  it  was  up  to  the  researcher  to  determine  which  category  a  response 
belonged  in.  Because  of  this,  conclusions  as  to  the  content  of  the  feedback  should  be 
made  with  caution.  Results  of  the  types  of  feedback  received  and  wanted  yielded  the 
percentages  listed  in  Table  7. 


Table  7.  Types  of  Feedback 


Type  of  Feedback  Received 

Type  of  Feedback  Wanted  j 

Group 

N 

Content 

Flow 

Mechanics 

N 

Content 

Flow 

Mechanics 

Online 

97 

51% 

33% 

16% 

62 

56% 

29% 

15% 

Face  to  Face 

84 

21% 

43% 

36% 

57 

32% 

46% 

23% 

Total 

181 

37% 

38% 

25% 

119 

45% 

37% 

18% 

The  online  group  reports  that  the  majority  of  the  comments  received  address  content. 
The  majority  of  comments  this  group  would  like  to  receive  also  address  content.  The 
face-to-face  group  reports  that  the  majority  of  the  comments  received  address  flow.  The 
majority  of  conunents  this  group  would  like  to  receive  also  address  flow. 


Summary 

Students  who  received  peer  feedback  did  not  report  more  improvement  in  writing 
skills  than  students  who  did  not  receive  peer  feedback.  All  students  who  participated  in 
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feedback  felt  that  feedback  was  worthwhile;  however,  students  who  participated  in  face- 
to-face  feedback  reported  a  significantly  higher  score  for  worth.  Students  who 
participated  in  online  peer  feedback  believed  that  LAKl  sessions  made  writing  more 
enjoyable,  while  students  who  participated  in  face-to-face  feedback  did  not  report 
agreement  with  this  belief.  The  analysis  shows  support  for  the  hypotheses  used  to  answer 
these  research  questions,  as  shown  in  Table  8. 


Table  8.  Results  Summary 


(  Research  Questions/Hypotheses 

Supported? 

RQ1 

Wni  Students  who  received  peer  feedback,  eitfier  online  or 
face  to  face,  report  more  improverr^nt  in  wnting  ^Ils  than 
students  who  did  not  parbdi^e  in  feedback? 

H1a 

Students  in  all  three  groups  will  report  a  significant  change  in 
their  writing  skills  between  the  beginning  of  the  semester  and 
ttie  end  of  the  semester 

Yes 

H1b 

Students  in  all  three  groups  will  report  improvement  in  their 
writing  skills 

Yes 

RQ2 

Will  students  who  partidpafe  in  online  peer  feedback  exhitA  . 
a  sbor^er  bd^  Biat  LAN  sessions  rnake  writir^  more 
«4^i>Ietten^ud^tsvi^parflCipate  infecetoface 
feedbMk?  ■  .-y  ■'  . 

'"'-‘I 

H2 

Both  the  online  feedback  and  ^ce  to  face  feedback  groups 
will  agree  with  the  statement  "LAN  sessions  make  wnting 
more  enjoyable." 

No 

Will  stuctents  vriho  parb'dpate  in  peer  feedbadt  repral  th^ 
feedback  Is  wor^while? 

I  H3 

the  mean  aggregate  score  for  the  items  that  measure  the 
concept  of  worth  will  be  higher  than  the  midpoint 

Yes 
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V.  Discussion 


Chapter  Overview 

This  chapter  offers  a  look  at  the  significance  of  the  results  found  in  Chapter  IV  in 
the  context  of  the  Air  Force  Academy  English  Department.  These  results  are  discussed 
as  they  reply  to  the  hypotheses.  Limitations  of  this  study  are  then  discussed,  and  finally, 
suggestions  for  further  research  are  provided. 

Discussion  of  Results 

In  this  section,  the  results  of  the  research  as  they  apply  to  each  research  question 
are  discussed.  Next  is  some  discussion  as  to  the  impact  these  results  may  have  on  the 
LAN  Online  Peer  Feedback  program. 

Research  Question  1.  It  was  not  surprising  to  see  that  the  students  reported  a 
significant  improvement  in  their  writing  skills  from  the  beginning  of  the  semester  to  the 
end.  It  would  have  been  unusual  for  them  to  report  no  improvement  or  negative 
improvement.  The  class  they  are  enrolled  in  is  designed  to  improve  their  writing; 
therefore,  it  was  expected  that  all  three  groups  would  report  a  significant  improvement  in 
their  writing  skills.  This  is  a  positive  sign  for  the  English  program  as  a  whole.  It 
suggests  that  the  course  is  serving  its  intended  purpose,  improving  the  writing  skills  of 
the  students. 

The  real  interest  lay  in  determining  if  the  groups  who  received  online  and  face-to- 
face  feedback  reported  significantly  more  improvement  than  the  group  that  received  no 
feedback.  It  was  expected,  based  on  the  literature,  that  the  students  who  received 
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feedback  would  report  more  improvement  than  the  students  who  received  no  feedback. 
Surprisingly,  this  was  not  the  case. 

While  the  overall  mean  improvement  for  the  no  feedback  group  was  lower  than 
the  mean  improvement  for  the  other  groups,  the  difference  was  not  statistically 
significant.  This  would  suggest  that  feedback  does  not  have  an  overall  impact  on  the 
perceptions  the  students  have  about  improvement  in  their  writing  skills.  Taken  by 
themselves,  these  results  could  be  an  argument  for  the  end  of  the  peer  feedback  program. 
However,  while  the  mean  improvement  for  the  no  feedback  group  was  not  significantly 
different  fi’om  the  mean  of  the  other  groups,  it  was  the  lowest  improvement  score.  It  is 
possible  that  with  further  study,  the  difference  could  prove  to  be  significant.  Therefore, 
an  argument  for  the  termination  of  the  peer  feedback  program  should  not  be  made  based 
on  this  data. 

Research  Question  2.  It  was  also  expected  that  all  students  who  participate  in 
peer  feedback  would  report  that  writing  was  more  enjoyable  when  the  LAN  was  used. 
The  online  feedback  group  did  report  a  significantly  higher  agreement  score  than  the 
face-to-face  feedback  group.  Surprisingly,  the  overall  mean  score  for  enjoyment  from  the 
face-to-face  feedback  group  was  2.932,  slightly  below  the  midpoint  of  the  scale.  This 
would  denote  an  attitude  that  is  leaning  toward  disagreement  with  the  statement.  This 
score  is  close  enough  to  the  neutral  score  to  be  considered  neutral,  but  is  not  the  attitude 
that  was  expected.  The  online  feedback  group’s  mean  score  is  on  the  “agree”  side  of 
neutral.  This  suggests  that  participating  in  both  writing  and  feedback  using  the  computer 
positively  affects  the  students’  enjoyment  of  the  writing  process. 
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Research  Question  3.  In  light  of  the  reported  improvement  in  writing  skill,  and 
the  agreement  that  LAN  sessions  made  writing  enjoyable  by  the  online  peer  feedback 
group,  it  was  not  surprising  that  students  who  participated  in  online  peer  feedback  found 
it  to  be  worthwhile.  It  was  surprising,  however,  to  find  that  the  students  who  participated 
in  face-to-face  feedback,  even  though  they  did  not  agree  that  LAN  sessions  made  writing 
more  enjoyable,  not  only  found  peer  feedback  to  be  worthwhile,  but  reported  a  stronger 
belief  that  it  was  worthwhile  than  the  online  feedback  group.  This  suggests  that  students 
need  not  find  writing  enjoyable  to  appreciate  a  tool  that  helps  them  improve  their  writing. 

Types  of  Feedback  Received.  According  to  the  literature,  students  prefer 
feedback  about  the  content  of  their  writing  as  opposed  to  feedback  that  simply  helps  with 
the  flow  or  corrects  mechanical  errors.  The  responses  to  the  questions  about  type  of 
feedback  received  and  type  of  suggestions  that  would  be  helpful  showed  some  surprising 
results,  suggesting  that  the  two  feedback  groups  receive  and  want  different  types  of 
feedback.  In  line  with  the  literature,  the  majority  of  the  online  feedback  group  receives 
and  wants  feedback  on  the  content  of  their  writing.  This  was  not  the  response  from  the 
face-to-face  feedback  group.  The  face-to-face  feedback  group  reported  data  that  suggests 
they  receive  and  want  feedback  relating  to  the  flow  of  their  writing.  The  unexpected 
aspect  of  this  result  is  that  the  two  groups  reported  a  desire  for  different  types  of 
feedback.  This  result  bears  further  consideration. 

Summary  of  Results.  The  overall  conclusion  gleaned  from  this  data  is  that,  even 
though  there  was  no  significant  difference  in  reported  improvement  in  writing  skills,  the 
peer  feedback  system  is  serving  a  useful  purpose.  One  of  the  groups  that  participated  in 


51 


peer  feedback  reported  that  it  made  writing  more  enjoyable  and  both  groups  that 
participated  in  peer  feedback  believed  that  it  was  worthwhile.  Given  the  findings,  one 
possible  suggestion  would  be  to  expand  participation  in  peer  feedback  so  that  every 
student  has  the  opportunity  to  use  this  worthwhile  tool.  The  reports  as  to  type  of 
feedback  received  and  wanted  suggest  that  the  two  feedback  groups  have  different  ideas 
about  the  purpose  of  feedback.  It  may  be  beneficial  to  conduct  a  review  of  the  feedback 
training  process  and  the  feedback  implementation  process  to  determine  where  this 
difference  in  ideas  comes  from.  Research  would  suggest  that  the  purpose  should  be  to 
improve  the  content  of  the  students’  writing.  The  students  should  be  given  an 
understanding  of  the  purpose  of  the  feedback. 

Limitations 

Several  limitations  in  this  study  made  data  comparison  difficult.  One  of  the 
limitations  stems  fi’om  the  fact  that  the  terms  and  conditions  used  for  the  peer  feedback 
program  were  defined  before  the  research  was  proposed.  These  terms  were  not  defined 
for  the  study  specifically.  Because  of  this,  there  was  a  lack  of  data  concerning  writing 
enjoyment  and  the  LAN  sessions  for  the  students  who  did  not  receive  feedback. 
Originally,  the  terms  online  feedback  and  LAN  session  were  used  interchangeably.  After 
careful  observation,  the  conclusion  was  reached  that  the  term  “LAN  session”  referred  to 
the  use  of  the  computer  in  general  and  was  not  associated  with  the  kind  of  feedback 
received.  As  a  result,  all  students  enrolled  in  English  111  should  have  been  considered 
participants  in  LAN  sessions.  All  students  participate  in  LAN  sessions,  but  not  all 
students  participate  in  feedback.  As  a  result,  question  11,  “LAN  sessions  make  writing 
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more  enjoyable,  did  not  just  apply  to  the  students  who  receive  peer  feedback.  It  would 
have  been  valuable  to  include  data  for  the  students  who  receive  no  peer  feedback  in  the 
analysis  of  this  question. 

The  second  limitation  of  this  study  was  the  reliance  on  perceptual  data.  While  it 
is  possible  to  determine  improvement  based  on  student  perception,  it  would  be  more 
valuable  to  determine  any  improvement  based  on  long  term  empirical  data  like  grades. 

The  third  limitation  of  this  study  was  the  relatively  small  number  of  subjects  in 
each  group.  While  the  groups  were  all  above  the  minimum  number  (30)  needed  to 
assume  a  normal  population,  the  small  number  limits  the  study’s  ability  to  be  reproduced 
in  larger  population  pools.  Future  study  using  a  larger  sample  size  should  be 
accomplished  in  order  to  verify  the  results  found  in  this  study. 

The  final  limitation  of  this  study  is  the  fact  that  the  study  itself  is  retrospective. 
This  IS  a  problem  for  two  reasons.  First,  the  way  the  data  was  collected  forced  the 
subjects  to  evaluate  performance  in  the  past  and  performance  in  the  present  at  the  same 
point  in  time.  This  may  have  affected  the  perceived  improvement.  Secondly,  the  study  is 
a  retrospective  report  on  situations  as  they  already  existed.  This  eliminates  any  ability  to 
control  the  variables.  In  the  future,  it  would  be  more  valuable  to  control  for  all 
independent  variables.  This  will  allow  the  differences  discovered  to  be  attributed  to  the 
dependent  variable  being  studied  (i.e.  kind  of  feedback).  These  controls  include  ensuring 
that  initial  feedback  instruction  is  completed  the  same  way  for  each  group,  and  ensuring 
that  partner  assignment  is  accomplished  the  same  way  for  each  group. 
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Recommendations  for  Future  Research 

As  stated  above,  it  would  be  valuable  to  include  the  students  who  receive  no  peer 
feedback  in  the  analysis  of  the  data  for  the  writing  enjoyment.  This  would  give  some 
indication  of  whether  the  reported  agreement  that  LAN  sessions  make  writing  more 
enjoyable  could  be  attributed  to  participation  in  LAN  sessions  as  a  whole,  or  participation 
in  peer  feedback  specifically. 

Further  study  needs  to  be  accomplished  to  determine  what  differences  are  caused 
by  the  different  types  of  feedback.  More  specifically,  while  the  improvement  reported 
was  not  significantly  different  for  the  three  groups,  the  no  feedback  group  had  a  mean 
that  was  lower  than  the  other  two  groups.  It  would  be  interesting  to  see  if  the  reported 
improvements  would  be  significantly  different  if  the  sample  group  contained  more 
members,  or  the  peer  feedback  system  had  been  in  for  a  longer  time.  It  would  also  be 
interesting  to  follow  a  set  of  students  for  a  period  of  years  to  see  of  long  term 
participation  in  peer  feedback  has  more  of  an  impact  on  writing  ability  than  no 
participation  in  peer  feedback. 

Future  research  should  also  be  accomplished  to  address  the  reported  differences 
between  the  online  group  and  the  face-to-face  group  in  feedback  received  and  wanted. 
Inquiry  into  the  validity  of  this  difference,  as  well  as  the  reason  for  it,  if  it  is  valid,  should 
be  accomplished. 

Future  research  should  address  the  limitations  of  the  questionnaire.  Despite  the 
reasonably  high  Cronbach’s  alpha  reported  for  the  items  that  measure  worth,  the 
questionnaire  was  a  preliminary  attempt  to  measure  the  opinions  of  feedback.  Future 
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studies  should  begin  with  a  more  complete  assessment  of  the  validity  of  the 
questionnaire,  and  progress  to  refinement  and  testing  of  the  questionnaire 

Finally,  future  research  should  focus  on  studies  that  overcome  the  limitations 
mentioned  above.  There  was  a  lack  of  empirical  studies  in  the  literature.  Future  research 
that  identifies  and  controls  for  the  variables  in  feedback  instruction  and  implementation 
would  be  a  valuable  addition  to  the  body  of  research. 

Thesis  Summary 

The  results  of  this  research  effort  provide  a  positive  statement  about  the  Air  Force 
Academy  English  Department  and  their  online  peer  feedback  program.  Students  report 
improvement  in  their  writing  skills  and  believe  that  peer  feedback  is  worthwhile.  This 
suggests  that  the  online  peer  feedback  program  is  effective  in  the  eyes  of  the  students. 
Also,  students  who  receive  online  peer  feedback  report  that  LAN  sessions  make  writing 
more  enjoyable.  This  provides  a  positive  argument  for  the  use  and  spread  of  the  peer 
feedback  program.  While  this  study  raises  more  questions  than  it  answers,  it  provides  a 
good  foxmdation  for  further  research  focused  on  the  identification  and  resolution  of 
questions  about  peer  feedback. 
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Think  about  the  peer  feedback  you  received  from  LAN  session  5  and  answer  the 
questions  below. 

1 .  What  kinds  of  suggestions  do  you  receive  from  your  classmates?  Please  give 
examples. 


2.  What  kinds  of  suggestions  are  most  helpful  to  you?  Please  give  examples. 


Please  pick  a  number  from  the  scale  to  show  how  much  you  agree  or  disagree  with  each 
statement  and  circle/mark  the  appropriate  number. 

3.  I  find  it  useful  to  have  my  classmates  read  my  papers  and  give  me  feedback 

1  2  3  4  5 

Strongly  Disagree  Neutral  Agree  Strongly 

Disagree  Agree 

4.  My  writing  is  likely  to  improve  because  of  peer  feedback 

1  2  3  4  5 

Strongly  Disagree  Neutral  Agree  Strongly 

Disagree  Agree 

5.  The  feedback  my  classmates  provide  is  valuable  to  me 

1  2  3  4  5 

Strongly  Disagree  Neutral  Agree  Strongly 

Disagree  Agree 


6.  Receiving  peer  feedback  helped  me  make  a  better  grade 

1  2  3  4  5 

Strongly  Disagree  Neutral  Agree  Strongly 
Disagree  Agree 
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7.  From  the  scale  below,  please  choose  the  number  that  best  describes  how  you  would 
rate  your  writing  skills  at  the  beginning  of  the  semester. 


average 


10 

excellent 


8.  From  the  scale  below,  please  choose  the  number  that  best  describes  how  you  would 
rate  vour  writing  skills  today. 


5  6 

average 


10 

excellent 


9.  From  the  scale  below,  please  choose  the  number  that  best  describes  how  you  would 
rate  the  writing  skills  of  the  entire  class  at  the  beginning  of  this  semester. 


5  6 

average 


10 

excellent 


10.  From  the  scale  below,  please  choose  the  niimber  that  best  describes  how  you  would 
rate  the  writing  skills  of  the  entire  class  as  of  today. 


ayerage 


10 

excellent 
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Please  pick  a  number  from  the  scale  to  show  how  much  you  agree  or  disagree  with  each 
statement  and  circle/mark  the  appropriate  number. 


1 1 .  LAN  sessions  make  writing  more  enjoyable. 


1 

2 

3  4 

5 

Strongly 

Disagree 

Neutral  Agree 

Strongly 

Disagree 

Agree 
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Appendix  B:  Questionnaire  for  No  Feedback  Group 


1 .  From  the  scale  below,  please  choose  the  number  that  best  describes  how  you  would 
rate  your  writing  skills  at  the  beginning  of  the  semester. 

123456789  10 

poor  average  excellent 

2.  From  the  scale  below,  please  choose  the  number  that  best  describes  how  you  would 
rate  vour  writing  skills  today. 

123456789  10 

poor  average  excellent 


3.  From  the  scale  below,  please  choose  the  number  that  best  describes  how  you  would 
rate  the  writing  skills  of  the  entire  class  at  the  beginning  of  this  semester. 

123456789  10 

poor  average  excellent 


4.  From  the  scale  below,  please  choose  the  number  that  best  describes  how  you  would 
rate  the  writing  skills  of  the  entire  class  as  of  today. 


1 

poor 


5  6 

average 


8 


10 

excellent 
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